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CHAPTER  1 


INTRODUCTION 


One  of  the  most  complex,  costly  and  controversial  aspects 
of  Federal  Government  procurement  is  the  selection  of  a  source 
for  award  of  a  contract  in  a  competitive  negotiated 
procurement.  Anyone  doubting  this  statement  need  only  peruse 
the  Government  procurement  decisions  of  the  Comptroller 
General  of  the  United  States  over  the  past  two  decades. 
Protests  against  the  award  of  negotiated  contracts  based  upon 
alleged  improper  source  selection  procedures  by  Federal 
procuring  activities  abound.  Yet,  it  is  not  difficult  to 
understand  why  such  protests  continue.  For  many  potential 
contractors,  the  award  of  a  Government  contract  could  mean  the 
very  survival  of  their  businesses.  Thus,  in  a  high-stafces , 
complicated  game,  disagreements  are  inevitable.  However,  the 
recurrent  nature  and  number  of  source  selection  protests  may 
indicate  that  the  basic  system  itself  needs  improvement. 

In  many  major  acquisitions,  agencies  prepare  detailed 
solicitations  which  purport  to  explain  the  Government's  needs 
to  industry,  even  though  they  may  not  really  be  certain  what 
these  needs  are,  or  when  they  will  be  required.  Contractors 
reciprocate  by  preparing  equally  detailed  and  expensive 
responses,  attempting  to  demonstrate  that  they  really  do 
understand  what  those  indefinite  needs  are  and  how  to  satisfy 
them  in  the  best  possible  manner. 
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Faced  with  the  task  of  deciding  which  response  in  reality 
satisfies  or  will  probably  satisfy  those  needs.  Government 
agencies  have  proved  quite  imaginative  in  their  attempts  to 
simplify  decision-making  processes  which  by  their  very  nature 
defy  simplification.  In  so  doing,  they  have  created  problems 
for  themselves,  perhaps  needlessy  in  some  instances,  and  the 
problems  persist.  Indeed,  it  has  been  seriously  suggested 
that  top  defense  contractors  not  compete  for  major  system 
contracts,  but  simply  be  awarded  them  on  a  rotational  basis.1 
This  review  will  attempt  to  examine  the  basic  elements  of 
source  selection  procedures  used  by  Federal  Government 
agencies,  with  the  dual  objectives  of  identifying  areas  of 
controversy,  and,  hopefully,  suggesting  possible  alternative 
solutions  to  those  problem  areas. 

Competitive  negotiated  procurement  normally  contains  the 
following  major  events: 

(a)  Identification  of  a  need  and  creation  of  a  plan  to 
satisfy  that  need. 

(b)  Communication  of  the  need  to  industry,  via  a 
solicitation.  This  is  usually  done  by  issuing  a 
Request  for  Quotations  (RFQ),  or  as  will  be  dealt 
with  most  in  this  review,  a  Request  for  Proposals 
( RFP ) • 

(c)  Evaluation  of  proposals  and  the  establishment  of  a 
competitive  range.  Those  proposals  found  to  fall 
outside  the  competitive  range  are  eliminated  from 
competition. 
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(d)  The  conduct  of  written  and/or  oral  dlcuaelons  with 
those  offerors  within  the  competitive  range. 

(e)  The  evaluation  of  best  and  final  offers  following 
discussions . 

(f)  Selection  of  the  offer  most  advantageous  to  the 
Government . 

(g)  Award  of  the  contract. 

Many  of  the  rules  which  concern  source  selection  procedures 
apply  to  all  phases  of  competitive  negotiation.  This  review 
will  not  consider  those  aspects  which  are  more  aptly 
categorized  as  negotiation  procedures)  such  as  establishment 
of  the  competitive  range,  and  the  conduct  of  oral  and  written 
discussions,  nor  will  it  consider  small  purchase  procedures* 
What  will  be  discussed  are  those  elements  of  negotiated 
procurement  which  directly  involve  the  source  selection 
process.  Three  distinct  areas  will  be  explored.  First, 
creation  of  a  source  evaluation  and  selection  system  will  be 
discussed.  This  entails  a  study  of  the  basic  structures  of 
various  agency  evaluation  systems,  including  its  personnel  and 
techniques  of  evaluation.  Next,  disclosure  of  the  particular 
system  is  examined.  This  involves  the  methods  chosen  to 
convey  the  agency's  needs  to  industry,  and  to  what  extent 
industry  is  and  should  be  made  aware  of  the  source  selection 
system.  Finally,  utilisation  of  the  chosen  system  is 
analysed.  Here  the  concern  is  the  extent  to  which  the 
disclosed  system  need  be  adhered  to  in  the  final  selection. 

Statutory  language  relative  to  source  silection 
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procedures  is  almost  nonexistent.  The  Armed  Services 
Procurement  Act,  10  U.S.C.  $§2301-2314,  contains  the  following 
language  in  §2304(g): 


In  all  negotiated  procurements  in  excess  of 
$10,000  in  which  rates  or  prices  are  not  fixed 
by  law  or  regulation  and  in  which  time  of 
delivery  will  permit,  proposals,  including 
price,  shall  be  solicited  from  the  maximum 
number  of  qualified  sources  consistent  with  the 
nature  and  requirements  of  the  supplies  or 
services  to  be  procured,  and  written  or  oral 
discussions  shall  be  conducted  with  all 
responsible  offerors  who  submit  proposals 
within  a  competitive  range,  price  and  other 
factors  considered:  Provided,  however,  that 
the  requirements  of  the  subsection  with  respect 
to  written  and  oral  discussions  need  not  be 
applied  to  procurements  in  implementation  of 
authorised  set-aside  programs  or  to 
procurements  where  it  can  be  clearly 
demonstrated  from  the  existence  of  adequate 
competition  or  accurate  prior  cost  experience 
with  the  product,  that  acceptance  of  an  initial 
proposal  without  discussion  would  result  in 
fair  and  reasonable  prices  and  where  the 
request  for  proposals  notifies  all  offerors  of 
the  possibility  that  award  may  be  made  without 
discussion . 


Obviously,  this  broad  statement  of  policy  leaves  room  for 
much  def initlzation  of  competitive  negotiation  procedures.  To 
tone  extent,  this  statutory  void  has  been  filled  by  agency 
regulations  and  by  the  decisions  of  the  Comptroller  General. 

As  such,  pertinent  regulations  of  three  military  and  three 
civilian  agencies  were  examined  in  preparation  for  this 
review,  as  were  the  decisions  of  the  Comptroller  General.  The 
Comptroller  General's  decisions  comprise  the  principal 
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decisional  materials  because  of  his  pre-eminence  in  the  area 
of  pre-award  protests.  Federal  courts  have  played  a  role  in 
the  pre-award  disputes  process,  but  it  has  proven  to  be  a 
limited  one.  Under  certain  circumstances,  an  aggrieved  bidder 
may  recover  the  costs  of  preparing  his  offer,  or  he  may  obtain 
injunctive  relief;  however.  Federal  Courts  will  usually  limit 
their  intrusion  into  the  source  selection  process  only  to 
situations  where  the  aggrieved  offeror  demonstrates  that  there 
was  no  rational  basis  for  the  agency's  decision,  and,  even 
where  no  such  basis  exists,  relief  may  nevertheless  be  refused 
if  an  overriding  public  interest  is  present.2  It  is  the 

Comptroller  General  who  has  been  the  guiding  force  in  the 

creation  of  most  of  the  specific  rules  which  have  filled  the 

statutory  void  over  the  past  two  decades  and  who  has  shaped 

the  structure  of  agency  regulations. 

Finally,  a  brief  introductory  note  about  the  regulations 
examined.  Regulatory  matters  having  general  application 
within  the  Department  of  Defense  (DOD)  include  the  Defense 
Acquisition  Regulation  (DAR),  1976  edition,  and  Department  of 
Defense  Directive  (DODD)  4105.62,  dated  January  6,  1976, 
entitled.  Selection  of  Contractual  Sources  for  Major  Defense 
Systems .  On  the  civilian  side,  the  Federal  Procurement 
Regulation,  (FPR),  1964  edition,  has  overall  application  to 
most  civilian  procuring  agencies.  The  National  Aeronautics 
and  Space  Administration  (NASA)  has  its  own  procurement 
regulations,  and  they  also  were  reviewed.  Some  procuring 
agencies  publish  detailed  guidance  on  competitive  source 
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selection  procedures.  On  the  Military  side,  all  of  the  DOD 
components  have  promulgated  regulations  which  implement  DODD 
4105.62.  The  latter  details  mandatory  policy  and  procedures 
on  competitive  source  selections  with  respect  to  high  dollar 
procurements.  As  the  title  Indicates,  the  Directive  is 
applicable  to  "major  defense  systems"  acquisitions,  i.e., 
those  programs  designated  by  the  Secretary  of  Defense  and 
having  sufficient  magnitude  to  justify  formalized  source 
selection  procedures.3 

Among  the  DOD  components,  the  Department  of  the  Air  Force 
appears  to  have  the  greatest  amount  of  regulatory  material 
specifically  devoted  to  the  source  selection  process.  The 
basic  Air  Force  regulation  is  Headquarters,  O.S.  Air  Porce 
Regulation  70-15  (APR  70-15),  dated  April  16,  1976.  It 

implements  DODD  4105.62  and  establishes  policy  and  procedures 
for  the  conduct  of  the  entire  competitive  source  selection 
process  of  major  systems.4  However,  for  those  procurements 
which  do  not  meet  the  dollar  thresholds  prescribed,  the 
regulation  specifies  that  the  provisions  therein  are 
sufficiently  flexible  to  accomodate  a  wide  range  of 
requirements,  and  thus  may  be  used  as  a  guide  to  formally 
evaluate  and  select  other  competitive  proposals  for  programs 
below  the  described  thresholds.5  In  addition,  the  Air 
Force's  major  subordinate  buying  commands.  Air  Porce  Systems 
Command  (APSC),  which  is  responsible  for  the  acquisition  of 
all  aircraft,  missile,  space  and  communications  systems,  and 
the  Air  Force  Logistics  Command  (APLC),  whose  responsibility 
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it  is  to  acquire  all  supplies  and  services  necessary  to 
support  those  systems  after  they  have  entered  the  Air  Force 
inventory,  have  supplemented  AFR  70-15  with  their  own 
regulations,  tailoring  the  specific  procedures  to  their  own 
needs  when  allowed  and  providing  guidelines  for  those 
procurements  which  fall  below  the  dollar  thresholds  of  major 
systems . 

The  basic  Army  regulation  is  Headquarters,  Department  of 
the  Army  Regulation  715-6  (AR  715-6),  dated  September  21, 

1970.  It  is  short  and  contains  only  broad  objectives  and 
policies,  and  it  too  is  applicable  to  high  dollar 
procurements.6  More  specific  guidance  in  source  selection 
procedures  is  found  in  a  pamphlet  published  by  the  Army's 
principal  purchasing  command.  Headquarters,  D.S.  Army  Materiel 
Development  and  Readiness  Command  (DARCOM),  formerly  known  as 
the  D.S.  Army  Materiel  Command.  DARCOM  is  responsible  for  the 
complete  life  cycle  of  Army  hardware,  from  research  and 
development  to  procurement,  production,  supply  and 
maintenance.  It  provides  supervisory  direction  to  seven 
subordinate  commands  in  procurement  matters.  DARCOM  Pamphlet 
715-3  (P715-3),  dated  October,  1980,  entitled  Proposal 
Evaluation  and  Source  Selection,  is  a  concise  handbook 
covering  the  entire  spectrum  of  the  source  selection  process. 
Though  not  mandatory,  the  guidance  provided  is  stated  to  be 
"based  upon  the  latest  experience  in  the  use  of  organised 
proposal  evaluation  and  source  selection  procedures",  and  is 
applicable  to  all  major  systems,  as  well  as  -lesser  dollar 
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acquisitions  of  high  complexity,  the  degree  of  application 
being  left  to  the  Source  Selection  Authority.7 

Procurement  for  the  Davy  is  conducted  by  five  principal 
decentralized  commands  under  Headquarters,  Naval  Material 
Command  (NAVMAT).  NAVMAT  is  a  supervisory  command,  and  it 
establishes  acquisition  policy  and  procedures  for  the 
subordinate  commands.  Hence,  the  basic  Naval  regulation 
concerning  competitive  source  selection  procedures  is  issued 
by  NAVMAT,  and  it  is  Known  as  NAVMAT  Instruction  4200.49, 
dated  February  28,  1977.  NAVMAT  Instruction  4200.49 

implements  DODD  4105.62  and  establishes  Chief  of  Naval 
Material  policy  and  guidance  on  source  selection  procedures 
for  the  acquisition  of  major  defense  systems.8  The  formal 
procedures  Included  in  the  Instruction  may  also  be  applied  to 
any  other  competitive  procurement  provided  advance  approval  is 
obtained  by  the  commander  of  the  particular  systems  command 
concerned.9  NAVMAT' s  subordinate  commands'  supplements  were 
also  examined;  they  include:  Naval  Electronics  Systems 
Command  (NAVELEX)  Instruction  4200.12c,  dated  August  31,  1978; 

Naval  Air  Systems  Command  (NAVAIR)  Instruction  4200.24,  dated 
July  1,  1977;  and.  Naval  Sea  Systems  Command  (NAVSEA) 
Instruction  4200.10,  dated  April  27,  1979. 

As  to  those  civilian  agencies  encompassed  by  the 
provisions  of  the  PPR ,  the  Department  of  Energy  (DOE)  has 
published  a  concise  handbook  called.  Procurement  Regulations 
Handbook,  Source  Evaluation  Board,  (DOE  PR-0027),  dated  June 
30,  1979.  Its  provisions  ars  mandatory  for  -major,  high  value 
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procurements  end  optional  in  others ,  and  they  may  be  modified 
to  fit  the  particular  procurement  as  long  as  any  such 
variations  do  not  constitute  a  departure  from  the  "basic 
policies  and  intent"  of  the  handbook.10  The  Department  of 
Health  and  Human  Services  ( HHS ) ,  formerly  the  Department  of 
Health,  Education  and  Welfare  (HEW),  has  relatively  little 
regulatory  guidance  on  source  selection  procedures  when 
compared  to  the  agencies  noted  above.  They  have  published 
their  own  procurement  regulations,  and  have  consolidated 
source  selection  matters  in  several  sections  of  an  internal 
handbook  entitled.  The  negotiated  Contracting  Process:  A 
Guide  for  Project  Officers,  1977  edition.  The  handbook 
generally  outlines  the  negotiated  procurement  process  from  its 
inception  in  the  advance  planning  stages  to  its  post  award 
administration  phase. 

The  last  regulatory  matters  examined  in  preparation  of 
this  review  belong  to  NASA.  In  addition  to  their  basic 
procurement  regulations,  NASA  has  provided  specific 
instructions  on  formal  source  selections  in  a  concise  manual 
entitled.  Source  Evaluation  Board  Manual  (NHB  5103. 6A),  1975 
edition.  The  policies  and  procedures  contained  therein  are 
mandatory  for  competitively  negotiated  procurements  when  the 
estimated  cost  of  the  contract,  together  with  the  estimated 
cost  of  later  phased  procurement  for  the  project  is  expected 
to  exceed  or  equal  $5  million,  as  well  as  to  any  other 
negotiated  procurement  which  a  source  selection  official 
determines  shall  be  evaluated  pursuant  to  the  manual.11 
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CHAPTER  II 

CREATION  OF  A  SOURCE  SELECTION  SYSTEM 

A.  Agency  Discretion 

Simply  stated,  the  object  of  negotiated  procurement  is  to 
select  that  proposal  which  is  most  advantageous  to  the 
Government.  In  making  these  selections.  Government  officials 
are  given  broad  discretion . 1 ^  indeed,  agency  discretion  is 
the  one  thread  that  binds  the  entire  source  selection  process 
together . 

A  classic  example  of  the  range  of  that  discretion  can  be 
found  in  a  1971  protest  involving  the  National  Aeronautics  and 
Space  Administration.13  It  might  be  worthwhile  tc  examine 
that  case  in  some  detail,  as  it  serves  as  a  good  introduction 
to  many  of  the  recurrent  problem  areas  in  the  selection 
process,  such  as  disclosure  of  evaluation  criteria  in  their 
relative  order  of  importance,  the  listing  of  subcriteria,  the 
utilization  of  numerical  scoring  and  weighting,  and  the 
discretion  of  the  source  selection  official  in  making  his 
decision.  In  that  case,  the  RCA  Service  Company  (RCA) 
protested  the  award  of  a  contract  for  institutional  support 
services  at  the  George  C.  Marshall  Space  Flight  Center.  Five 
main  evaluation  criteria  were  to  be  used  in  evaluating 
proposals,  and  the  RFP  set  forth  their  relative  importance  in 
the  following  manneri 
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B.  Evaluation  Criteria  -  Proposals  will  bs  evaluated 
and  ranked  against  the  following  criteria  and  a 
numerical  score  assigned.  The  total  weights  of  the 
first  three  criteria  are  approximately  five  times 
greater  than  the  weight  of  the  fourth  criteria.  In 
evaluating  the  first  three  criteria,  primary 
consideration  will  be  given  information  received 
under  the  first  two  criteria  (Management  Plan  and 
Previous  Experience),  which  are  approximately  equal 
to  each  other.  Information  received  under  the  third 
criterion  (Staffing  Plan)  will  be  given  somewhat  less 
consideration  than  either  of  the  first  two  criteria. 
In  evaluating  the  last  two  criteria,  primary 
consideration  will  be  given  information  received 
under  the  fourth  criterion  (Policies,  Procedures  and 
Financial  Capability).  Significantly  less 
consideration  will  be  given  the  fifth  criterion 
(Facilities  and  Equipment).14 


The  RFP  then  proceeded  to  describe  the  important  aspects 
of  the  criteria,  listing  several  subcriteria  under  each  major 
criterion.  Criteria  1  and  2  contained  the  following 
information  pertinent  to  the  protest: 


1 .  Management  Plan 

*  *  •  *  * 

( f )  Small  and  Minority  Business  Utilisation 
Plan 

-  Special  consideration  will  be  given  to 

proposals  containing  firm  commitments  to 
small  business  subcontractors  on  minority- 
owned  enterprises. 

***** 

2.  Previous  Experience  -  Evaluation  of  your 
previous  experience  will  include  the  extent  to 
which  directly  related  services  have  been 
successfully  performed  end  managed  during  the  past 
several  years. 
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The  RPP  expanded  the  above-cited  language  In  other  sections, 
generally  requiring  information  in  the  scope  of  work 
prospective  contractors  contemplated  would  be  performed  bv 
subcontract  under  set-aside  arrangements  for  small  business  or 
minority-owned  concerns,  and,  if  an  offeror  did  not  to 
contemplate,  information  on  his  reasons  therefor.  As  to 
previous  experience.  Section  II  of  the  RFP  required  the 
offeror  to  provide  a  listing  of  related  technical  experience 
and  contracts  both  he  and  his  proposed  subcontractors  had 
performed,  and  was  encouraged  to  furnish  a  self-analysis  of 
such  previous  experience. 

Proposals  were  received  from  three  offerors,  Hayes 
International  Corporation  (Hayes),  RCA,  and  the  Federal 
Electric  Corporation  (PEC).  Those  proposals  were  analysed  in 
accordance  with  an  evaluation  plan  established  prior  to 
issuance  of  the  RFP.  In  the  plan,  the  five  major  criteria 
were  assigned  numerical  weights  indicative  of  their  relative 
importance.  However,  the  subcriteria  were  not  to  be 
numerically  scored,  but  rather  were  to  be  Informally 
evaluated.  Following  initial  evaluation  and  oral  discussions 
(where,  pursuant  to  NASA  directives,  deficiencies  were  not 
disclosed),  RCA  was  ranked  first  with  a  final  weighted  score 
of  890  out  of  a  possible  1000,  FEC  second  with  an  840  score, 
and  Hayes  third  with  a  score  of  800.  Xn  their  evaluation,  the 
members  of  the  Source  Evaluation  Board  (SEE),  noted  that  while 
Hayes  had  limited  experience  in  several  key  aspects  of  the 
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Previous  Experience  criterion,  they  did  propose  to  subcontract 
the  custodial  services  to  a  newly  organized  minority  firm. 

In  his  decision,  the  Source  Selection  Official  (SSO) 
concluded  that  the  competition  was  close,  and,  in  spite  of  the 
fact  Bayes  was  ranked  last,  chose  that  company  for  award.  He 
stated  that  the  differences  in  the  proposals  were  not  great, 
that  any  weaknesses  in  Hayes'  proposal  appeared  to  be 
correctable,  and  that  he  was  particularly  impressed  by  the 
effort  of  Hayes  in  locating  and  assisting  a  minority-owned 
firm  for  such  an  important  subcontract.  Further,  he 
discounted  any  weaknesses  of  Hayes  as  a  result  of  the 
subcontractor's  lack  of  experience  as  an  "unavoidable 
consequence  of  increasing  minority-owned  business 
participation." 

Following  the  selection,  the  "losers"  were  then 
debriefed.  As  is  the  usual  case,  it  is  here  where  the 
frustrated  offeror  gets  his  first  glimpse  of  the  actual, 
detailed  process  of  evaluation  and  selection.  In  its 
debriefing,  RCA  was  criticized  for  token  utilization  of 
minority-owned  concerns.  Undoubtedly  to  no  one's  surprise, 

RCA  protested  to  the  Comptroller  General,  alleging  essentially 
that  the  880  gave  greater  weight  to  the  small  and  minority 
business  subcriterion  than  was  otherwise  disclosed, 
particularly  in  view  of  its  being  placed  last  under  the 
general  criterion!  and,  further,  that  the  decision  to  discount 
the  reported  weakness  of  Hayes  flowing  from  the  lack  of 
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experience  of  that  company's  minority  subcontractor  was  in 
contravention  of  the  RFP  statement  that  previous  experience, 
including  that  of  subcontractors,  was  an  important  evaluation 
criterion . 

The  Comptroller  General  agreed.  He  reiterated  his 
consistent  position  that  evaluation  criteria  and  their 
relative  importance  should  be  disclosed  to  offerors,  and  cited 
NASA  regulations  to  that  effect.  He  stated  the  action  of  the 
SSO  in  discounting  the  inexperience  of  Hayes'  minority  group 
subcontractor  reflected  a  different  interpretation  of  the 
"Previous  Experience"  criterion  than  that  utilized  by  the  SEB, 
and  the  only  reasonable  interpretation  which  could  be  derived 
from  the  RFP.  In  addition,  the  Comptroller  General  noted  no 
indication  was  given  to  offerors  relative  to  the  extent  to 
which  special  consideration  was  to  be  given  to  commitments  to 
small  or  minority-owned  enterprises,  or  that  more 
consideration  would  be  given  to  commitments  to  small  versus 
minority-owned  concerns,  or  vice-versa.  While  agreeing  there 
is  no  obligation  to  advise  bidders  of  the  relative  importance 
of  subcriteria,  or  to  list  such  subcriteria  in  descending 
order  of  importance  if  they  are  to  be  considered  of 
approximate  equal  importance,  the  Comptroller  General  stated 
that  where  one  such  subcriterion  possesses  overriding 
importance,  offerors  should  be  so  advised.  In  the  absence  of 
such  advice,  offerors  are  entitled  to  assume  that  all 
subcriteria  will  be  considered  of  approximately  equal 
importance.  Here,  the  actions  of  the  SSO  clearly  demonstrated 
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that  Hayes'  utilization  of  a  Minority-owned  business  was  a 
primary  factor  in  selection  for  award.  While  noting  the  SSO 
has  the  right  to  change  weights,  when  this  occurs  offerors 
should  be  informed  and  allowed  an  opportunity  to  submit 
revised  proposals. 

Based  upon  the  above,  the  Comptroller  General  concluded 
that  the  actions  of  the  SSO  constituted  a  departure  from 
"sound  procurement  policy."17  However,  notwithstanding  these 
improprieties,  this  departure  was  not  sufficient  to  invalidate 
the  selection.  His  reasoning  was  as  follows: 


We  cannot  state  as  a  matter  of  law  as 
distinguished  from  sound  procurement  policy, 
that  your  Administrator  is  without  authority  to 
make  an  award  of  this  contract  to  Hayes.  The 
RFP  did  specify  that  special  consideration 
would  be  given  to  proposals  evidencing  fliJi 
commitments  for  minority  subcontracting,  albeit 
in  terms  that  were  not  sufficiently  precise  to 
warrant  clearly  the  action  taken.  All  of  the 
proposals  were  meticulously  and  fairly 
evaluated  by  the  SEB  according  to  the  precise 
terms  of  the  RFP. 

*  *  * 

There  can  be  no  question  but  that  the  basis 
upon  which  an  award  to  Hayes  is  being  proposed 
has  been  openly  stated,  with  no  implication 
that  the  Source  Selection  Official  in  making 
his  choice  is  proceeding  in  other  than  a 
straightforward  manner  after  full  and  fair 
evaluation  of  all  proposals.  While  there  may 
be  grounds  for  disagreement  with  the  reasoning 
by  which  the  Source  Selection  Official  chose  to 
select  Hayes  over  the  other  offerors,  we  cannot 
conclude  that  such  selection  was  arbitrary.  In 
view  thereof,  and  recognizing  the  absence  of 
either  a  statutory  or  regulatory  direction 
relative  to  awards  in  negotiated  procurements 
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of  this  type,  we  cannot  conclude  that  the 
departure  from  what  we  consider  to  be  sound 
procurement  policy  from  a  competitive 
standpoint  is  sufficient  in  itself  to  preclude 
the  Source  Selection  Official  from  making  his 
selection  on  the  basis  stated. 

Thus,  this  broad  agency  discretion  apparently  extends  to 
breaches  of  sound  procurement  policy,  as  long  as  the  decision 
is  not  arbitrary.  Nevertheless,  one  might  wonder  if  the  award 
to  Bayes  was  "most  advantageous"  to  the  Government.  Perhaps 
it  can  be  so  only  if  the  accomplishment  of  recognized  social 
objectives,  i.e.,  the  betterment  of  minority-owned  concerns, 
can  be  accepted  as  a  legitimate  advantage. 

For  a  judicial  discussion  of  the  limits  of  agency 
discretion  by  a  Federal  District  Court  in  its  limited 
pre-award  role,  see  Rudolph  F.  Matzer  &  Associates,  Inc,  v. 
Warner , 19  where  the  court  noted  its  duty  to  interfere  in  the 
procurement  process  when  agency  officials  exercise  their 
discretion  "in  an  abusive,  unlawful  or  irrational  manner,"  and 
stated  examples  of  such  behavior  would  be  where  an  applicable 
statute  or  regulation  was  violated,  or  where  the  record 
contained  no  substantial  evidence  on  which  the  agency  official 
could  base  his  decision. 

Again,  broad  agency  discretion  can  be  found  in  all  phases 
of  the  source  selection  process.  This  chapter  will  examine 
that  discretion  as  it  relates  to  the  creation  of  a  source 
selection  system. 


if 
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B .  Determination  of  Minimum  Heeds 


Agency  discretion  perhaps  enjoys  its  greatest  latitude  in 
the  initial  phase  of  the  creation  of  a  source  selection 
system,  or  indeed  in  the  initial  step  of  any  procurement. 

That  initial  step  is  simply  the  identification  of  the  minimum 
requirements  or  needs  that  will  satisfy  the  Government's 
desires.  In  that  regard,  it  should  be  recognized  that  minimum 
need  determinations  really  involve  restrictive  competition 
issues.  In  essence,  the  ultimate  consideration  here  is 
whether  or  not  the  specified  requirements  unduly  restrict 
competition.  Minimum  needs,  as  they  relate  to  competition, 
really  encompass  restrictions  which  bear  some  relevance  to  the 
satisfaction  of  the  desired  product  or  service.  That  is,  the 
term  "minimum"  does  not  mean  the  Government  must  always  settle 
for  only  those  requirements  absolutely  necessary  in  order  to 
function.20  Indeed,  certain  minimum  needs  may  require  the 
best  available  services  or  products.21  In  any  case,  it 
should  be  remembered  that  the  minimum  need  considerations 
discussed  herein  also  concern  and  ultimately  involve  such 
restrictive  competition  issues. 

The  Comptroller  General  has  on  many  occasions  reaffirmed 
his  position  that  the  determination  of  the  Government's 
minimum  needs  is  best  done  by,  and  the  primary  responsibility 
of,  those  who  are  most  familiar  with  the  conditions  under 
which  those  needs  are  to  be  used,  i.e..  Government  procurement 
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officials.22  His  office  will  not  question  agency  minimum 
need  decisions  unless  clearly  shown  to  be  unreasonable,  an 
arbitrary  abuse  of  discretion,  or  a  violation  of  a  procurement 
statute  or  regulation.23  The  protestor  has  the  burden  of 
affirmatively  proving  unreasonableness.2*  This  is  a  heavy 
burden,  as  numerous  protestors  have  discovered,  particularly 
where  the  procurement  involves  equipment  of  a  highly  technical 
or  scientific  nature  and  the  determination  must  be  based  on 
expert  and  technical  opinions.26  For  example,  HEW  easily 
justified  their  need  for  more  advanced  characteristics  than 
those  found  in  commercially  available  scientific  equipment 
through  the  opinions  of  individual  research  microbiologists, 
for  whom  the  instruments  were  to  be  procured.26 

However,  the  Comptroller  General  has  on  occasion 
criticized  stated  agency  requirements.27  In  one  recent 
case,  the  Environmental  Protection  Agency  set  forth  stringent 
experience  requirements  for  the  acceptability  of  maintenance 
personnel.28  The  protestor  alleged  that  the  requirements 
were  excessive,  and  that  they  were  improper  in  using  years  of 
experience  as  the  sole  criterion,  which  gave  no  credit  for 
education  and  training.  The  Comptroller  General  stateds 
Iwjhile  not  deciding  the  issue,  we  believe,  in  the  interest 
of  furthering  competition,  that  the  KPA  should  review  the 
experience  requirements  with  a  view  to  reducing  them  *  *  *  or 
accepting  equivalent  education  and  training  to  fulfill  a 
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portion  of  the  requirement."29  Nevertheless,  the  protest  was 
denied. 

Admittedly,  these  latter  cases  are  rare,  and  it  is  clear 
that  the  broad  discretion  "thread"  dominates  the  establishment 
of  minimum  needs  by  the  respective  Government  agencies. 

C.  Development  of  Evaluation  Criteria 

1 .  Common  Types  of  Criteria 

Once  those  minimum  needs  are  determined,  it  becomes 
necessary  to  express  those  needs  in  an  understandable  manner. 
In  the  source  selection  process,  those  expressions  are 
commonly  called  evaluation  criteria  or  evaluation  factors. 
Whether  they  are  labeled  as  elements,  factors,  or  some  other 
name,  these  things  comprise  the  basic  aspects  of  the  needs 
which  are  essential  to  their  fulfillment. 

The  basic,  general  areas  of  evaluation  criteria  appear 
in  most  programs.  Usual  major  categories  include  Technical, 
Management,  Experience  and  Past  Performance,  Cost/Price,  and 
other  areas  when  appropriate.  The  various  risks  associated 
with  proposals  in  relation  to  major  factors,  especially 
Technical,  may  be  also  disclosed  as  evaluation  factors  in  many 
procurements.  Some  of  the  regulations  attempt  to  give 
guidance  in  the  choice  of  major  factors  and  subfactors. 

The  Air  Force  has  described  in  some  detail  how  it  expects 
their  buying  commands  to  establish  evaluation  criteria.  AFR 
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70-15  provides  specific  guidance  on  evaluation  criteria  to  be 
selected  and  disclosed.  Normally,  criteria  are  classified 
into  general  and  specific  considerations . 30  General 
considerations  include  such  things  as  past  performance, 
contractual  terms  and  conditions,  and  results  of  preaward 
surveys.  Specific  criteria  form  the  basis  for  the  technical 
evaluation  and  are  classified  as  Areas,  Items  and  Factors.31 
"Areas'*  are  basic  functional  disciplines  containing  elements 
that  will  have  an  impact  on  the  success  of  the  program  and  the 
selection  process,  the  most  generally  encountered  areas  being 
technical,  operations,  logistics,  management/production  and 
costs.32  Technical  risk,  as  it  pertains  to  each  proposal,  is 
an  evaluation  criterion,  and  is  rated  based  on  the  "offeror's 
risk  assessment  and  the  credibility  of  his  proposed  approach 
for  eliminating  or  avoiding  such  risks."33  Obviously,  a  more 
specific  breakdown  is  necessary  for  proper  analysis  and  those 
breakdowns  are  called  "Items."  An  illustration  of  such  a 
breakdown  can  be:  Area-Operations,  Item-Maneuverability.34 
Each  of  the  Items  of  the  Areas  determined  to  be  essential  to 
the  selection  process  may  nevertheless  still  be  fairly  broad 
in  scope.  As  such,  further  segmentation  may  be  necessary,  and 
those  segmentations  are  called  "Factors”  by  AFR  70-15.  Only 
in  rare  instances  is  segmentation  below  Factor  level 
contemplated.  An  AFR  70-15  example  of  a  Factor  shradout 
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Area  -  Operations 

Item  -  Maneuverability 
Factor  -  Turn  Radius 
Factor  -  Excess  Power 
Item  -  Survivability 

Factor  -  Subsystem  Redundancy 
Factor  -  Radar  Cross  Section 
Subfactor  -  Front  Quarter 
Subfactor  -  Side  View35 

The  establishment  and  basic  struccure  of  criteria  and 
subcriteria  in  the  Army  are  similar  to  that  of  the  Air  Force. 
Evaluation  criteria  are  segmented  into  three  basic  levels 
called:  areas,  elements  (items  in  Air  Force  terminology)  and 

factors.36  DARCOM  guidance  notes  that  technical,  cost  and 
management  are  traditional  major  areas,  while  others  might  be 
logistics,  operational  suitability  and  test  evaluation. 37 
Those  Naval  regulations  studied  for  this  review  provide  no 
substantive  guidance  on  the  selection  or  format  of  evaluation 
criteria . 

As  to  the  civilian  agencies  examined,  HHS  simply  provides 
that  evaluation  criteria  are  to  be  developed  by  technical 
personnel,38  while  DOE  notes  only  that  such  criteria 
generally  fall  into  three  major  categories:  technical, 
business  and  management,  and  cost.38  However,  NASA  describes 
four  basic  groups  of  evaluation  factors  in  most  of  their 
procurements  as: 

A.  "Mission  Suitability  Factors."  These  include 
both  technical  and  management  considerations  and 
they  are  numerically  scored  and  weighted.40 
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B.  "Cost  Factors."  These  are  not  scored,  because 

the  "weight  to  be  accorded  thea  can  be  judged  by 
the  SSO  only  after  he  has  determined  the 
relative  merits  of  the  proposals  from  a  mission 
suitability  standpoint  and  the  significance  of 
differences  in  this  regard,  and  after  he  has 
adjudged  the  significance  of  Other  Factors."41 

C.  "Experience  and  Past  Performance."  These  factors 

are  not  numerically  scored  either.42 

D.  "Other  Factors."  That  is,  those  other 

considerations  which  have  relevance  to 
the  particular  contract,  such  as  labor 
relations,  small  business  considerations,  etc. 
These  factors  are  also  not  numerically  scored, 
but  are  subjectively  evaluated,  like  factors  2 
and  3  above. 4^ 

2 .  Cost/Price  as  a  Criterion 

Logically,  cost/price  is  always  a  factor.44 
Only  the  relative  importance  of  cost/price  changes,  depending 
upon  the  nature  of  the  acquisition  and  the  type  of  contract 
Involved.  For  example,  in  a  research  and  development 
contract,  cost  will  usually  be  of  less  importance  than 
technical  excellence.  Zn  cost  reimbursement  contracts,  the 
realistic  expected  cost  of  performance,  normally  reflected  in 
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Government  estimates/  is  used  to  evaluate  proposals/  and  not 
the  offeror's  estimated  cost.45  Heedless  to  say,  the  offeror 
is  entitled  to  know  the  relative  importance  of  cost/  and  how 
it  fits  into  the  evaluation  process.  The  relationship  of 
cost/price  with  other  factors  and  the  evaluation  thereof  are 
discussed  in  detail  in  later  sections  of  this  review. 

Other  types  of  costs  are  sometimes  used  as  evaluation 
factors  when  appropriate  and  when  listed  in  the  solicitation. 
Common  examples  are  phase-in  costs46  and  costs  of 
ownership.47  DOD  major  systems  acquisitions  contain  the 
evaluation  of  life  cycle  costs.48  Life  cycle  cost  is  the 
total  cost  of  an  item  over  its  full  life,  including  the  cost 
of  development,  acquisition,  ownership  and,  where  applicable, 
the  cost  of  disposal.49  It  might  be  noted  here  that  when 
life  cycle  costs  are  to  be  evaluated,  the  solicitation  should 
indicate  that  fact  as  well  as  the  useful  life  period  to  be 
used  in  the  evaluation.50 

3 .  Past  Performance  as  a  Criterion 


Another  evaluation  criterion  which  has  increased  in 
use  in  the  past  years  and  appears  in  most  major  acquisitions 
is  past  performance.  All  of  the  agencies  examined  mentioned 
the  use  of  past  performance  as  a  factor  when  appropriate. 
DODD  4105.62  requires  that  DOD  components  utilise  past 
performance  when  relevant  to  the  contract.51 


NASA  lists  past 
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performance  as  a  basic  factor  normally  present  in  most 

procurements . 52  DOE  considers  it  under  their  "business  and 

management"  criterion, 5^  and  BBS  evaluates  past  performance 

as  part  of  an  offeror's  business  proposal  in  each  contract.54 

In  the  Air  Force,  AF SC  attempts  to  define  past  performance  as: 

[R]elevant  past  performance  means  quality  of 
work,  essentially  comparable  to  the  instant 
acquisition,  completed  under  and  in  accordance 
with  a  contract.  It  includes  but  is  not 
necessarily  limited  to  work  in  the  same  or 
similar  acquisition  phase  or  category,  for  the 
same  or  similar  item,  and  of  a  similar  scope, 
performed  by  the  same  company/division  profit 
center,  and  in  a  time  period  reasonably  recent 
to  the  instant  acquisition.55 


Further,  AFSC  considers  past  performance  to  be  important  in 
most  cases,  as  their  regulation  states:  "The  past  performance 

criterion  shall  be  equal  in  ranking  or  stature  to  all  other 
criteria  if  all  are  equal,  or  first  if  ranked  in  order  of 
importance. " 56 

Fast  performance  factors  are  also  elements  in  the 
determination  of  an  offeror's  responsibility,  such 
determination  being  made  either  by  the  contracting  officer  or 
by  the  Small  Business  Administration  via  their  Certificate  of 
Competency  procedure  for  small  businesses.  If  so,  the  obvious 
question  then  is  what  business  does  an  evaluation  board  or 
team  have  in  considering  those  elements  during  the  evaluation 
phase.  The  Comptroller  General  has  answered  that  question  on 
several  occasions.  Simply  stated,  criteria  which 
traditionally  related  to  responsibility,  i.e.,  an  offeror's 
ability  or  capacity  to  perform,  may  raise  questions  of 
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technical  acceptability  in  the  context  of  negotiated 

procurements.57  In  one  case,  HEW  determined  that  a  small 

business  offeror  was  technically  unacceptable  primarily 

because  of  a  low  rating  given  in  the  corporate  experience 

area.®8  The  RFP  clearly  indicated  that  information  on  such 

experience  was  required  and  would  be  evaluated.  The  offeror 

protested  that  he  was  rejected  for  reasons  bearing  on 

responsibility,  and  that  it  was  improper  to  do  so  at  that 

time.  The  Comptroller  General  disagreed,  stating: 

In  many  other  cases,  we  have  recognised  that 
contracting  agencies  may  properly  utilise 
evaluation  factors  which  include  experience  and 
other  areas  that  would  otherwise  be  encompassed 
by  offeror  responsibility  determinations  when  the 
needs  of  those  agencies  warrant  a  comparative 
evaluation  of  those  areas.59 

Accordingly,  no  impropriety  was  found  in  the  use  of  experience 
factors  as  proposal  evaluation  criteria,  or  in  HEW's  rejection 
of  the  proposal  without  referring  the  matter  to  the  Small 
Business  Administration.  Thus,  when  their  minimum  needs 
require  a  comparative  analysis  of  offeror's  experience  or 
other  responsibility  type  areas,  contracting  agencies  may 
utilise,  in  negotiated  procurements,  evaluation  factors  which 
measure  such  areas.50 

Finally,  it  might  be  mentioned  that  the  experience  of 
proposed  subcontractors  may  be  used  in  determining  whether  a 
prime  contractor  meets  the  solicitation's  experience 
requirements.51  Also,  where  responsibility-type  criteria  are 
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comparatively  evaluated,  the  rule  that  responsibility 
determinations  should  be  based  on  the  most  current  information 
available  is  likewise  applicable  to  such  comparative 
evaluations. 6^ 

4.  Other  Considerations 


Whatever  the  manner  or  category  preferred,  the  choice 
of  evaluation  factors  is  obviously  a  matter  of  common  sense. 
The  specific  number  and  types  will  naturally  depend  upon  the 
particular  procurement.  Logically,  the  listing  of  too  many 
car.  be  just  as  harmful  as  listing  too  few,  since  the  former 
may  confuse  while  the  latter  invariably  might  lead  to 
misunderstandings.  In  any  case,  they  should  be  as  precise  and 
as  specific  as  possible,  since  factors  stated  in  general  terms 
could  cause  the  solicitation  to  be  defective.63  Clearly, 
they  should  include  all  significant  aspects  which  will  have  an 
impact  on  the  ultimate  decision. 

D .  Personnel  and  Procedures 

1 .  General  Considerations 


As  procurements  become  more  complex,  so  do  the 
evaluation  structures  supporting  them.  All  of  the  regulations 
noted  previously  contemplated  the  establishment  of  formal 
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•valuation  boards  or  teams  to  rate  and  evaluate  proposals. 
While  agencies  again  enjoy  wide  discretion  in  the  use  of  such 
boards/  the  basic  framework  of  the  various  agency  systems  is 
similar/  particularly  for  major  acquisitions. 

For  example,  DOD  components  use  essentially  identical 
evaluation  structures  for  major  system  acquisitions.  That 
structure  consists  of  a  decision  pyramid  made  up  of  a  Source 
Selection  Evaluation  Board  (SSEB)  at  the  base,  a  Source 
Selection  Advisory  Committee  (SSAC)  in  the  middle,  and  a 
Source  Selection  Authority  (SSA)  at  the  top.64  The  SSEB 
consists  of  a  group  of  personnel  representing  the  various 
functional  and  technical  disciplines  necessary  for  the 
particular  procurement,  who  are  appointed  by  the  SSAC  to 
evaluate  proposals  and  develop  summary  facts  and  findinqs. 

The  SSEB  issues  an  Evaluation  Report,  which  contains  detailed 
narrative  assessments  of  each  proposal  against  evaluation 
standards,  numerical  scores  when  used,  and  summary  appraisals 
of  significant  strengths,  weaknesses  and  risks  of  each 
proposal. 65  The  SSAC  is  usually  a  group  of  senior  military 
or  Government  civilian  personnel  appointed  to  advise  the  SSA 
during  the  source  selection  process.  They  analyse  the  results 
of  the  SSEB's  evaluations  and  prepare  an  Analysis  Report.66 
The  latter  is  a  formal  report  which  analyses  the  findings  of 
the  S8EB,  summarises  the  entire  process  to  date,  and  draws 
conclusions,  but  normally  makes  no  recommendations.  As 
required  by  DODD  4105.62,  the  SSA  is  the  official  designated 
to  direct  the  entire  process  and  select  the  source(s).67 
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Scaled  down  procedures  may  be  used  for  small  procurements, 
such  as  the  NAVSEA  "CARP",  or  Contract  Award  Review  Panel, 
which  functions  as  a  combined  SSAC  and  SSEB.68  Below  certain 
projected  dollar  values  for  specific  contracts,  formalised 
procedures  are  usually  neither  warranted  nor  necessary.  AFSC 
specifies  in  their  supplement  to  APR  70-15  that  formal  source 
selection  procedures  are  discouraged  for  procurements  expected 
to  cost  less  than  $1  million.69 

On  the  civilian  side,  DOE  utilises  an  evaluation  and 
selection  structure  consisting  of  a  Source  Evaluation  Board 
(SEB)  and  a  Source  Selection  Official  (SSO).70  Again,  when 
the  complexity  and  value  of  the  procurement  so  warrants,  this 
formal  structure  will  be  used.  The  duties  and 

responsibilities  of  the  SSO  are  similar,  if  not  identical,  to 
that  of  the  SSA  in  the  military  agencies,  while  the  SEB’s 
tasks  resemble  a  combined  SSAC  and  SSEB  in  the  military 
structure.  In  HHS,  proposals  are  required  to  be  in  two  parts, 
a  "technical  proposal"  and  a  "business  proposal".71 
Technical  proposals  are  simply  provided  to  "technical 
evaluators",  who  prepare  an  evaluation  report  for  the 
Contracting  Officer.72  Finally,  NASA's  formalized  system 
consists  of  an  SSO  and  SEB,  similar  to  the  structure  of 
DOE.73  The  SEB  is  responsible  for  the  solicitation,  receipt 
and  evaluation  of  proposals,  and  for  establishing  evaluation 
critaria  and  the  requisite  weights.  When  needed,  the  SEB  may 


be  supported  by  various  committees  who  serve  es  fact-finding 
bodies  for  their  related  functional  disciplines;  however,  the 
SEB  is  required  to  use  its  own  collective  judgment  in  arriving 
at  its  conclusions  for  purposes  of  reporting  to  the  SSO.74 

Even  though  agencies  may  utilize  formalized  evaluation 
boards  and  procedures,  there  is  no  legal  requirement  that  they 
do  so.  The  Comptroller  General  has  sanctioned  evaluations 
conducted  by  one  person,  if  properly  qualified,75  and  by 
outside  review  committees  who  submit  their  findings  to  agency 
evaluation  personnel.76  Procedural  aspects  of  proposal 
evaluation  are  matters  requiring  agency  discretion,  and  won't 
be  questioned  absent  clear  evidence  of  abuse.77  Thus,  where 
a  large  number  of  proposals  must  be  considered,  it  is  not 
improper  or  arbitrary  to  divide  them  among  panel  members  for 
evaluation.  In  that  regard,  one  protest  challenged  the  basic 
fairness  of  a  procedure  whereby  30  proposals  were  divided 
among  evaluators,  and  any  proposal  could  be  rejected  by  only 
one  person.78  In  denying  the  protest,  the  Comptroller 
General  stated  that  where  the  RPP  doesn't  provide  that  each 
proposal  will  be  rated  by  all  evaluators,  the  fact  that  some 
would  be  read  by  only  one  evaluator  provided  no  grounds  for 
relief.  Similarly,  an  HHS  requirement  that  technical 
evaluators  submit  narrative  evaluations  with  their  raw  scores 
didn't  require  a  narrative  from  each  evaluator;  rather,  a 
consensus  report  containing  such  narrative  reasoning 
sufficed.79  Also,  some  procuring  agencies  do  not  provide 
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their  technical  evaluators  with  cost  information.80  Such  a 
procedure  is  discretionary,  as  the  Comptroller  General  has 
stated  there  is  no  law  which  would  be  contravened  by  such  a 
disclosure . 8 1 

Some  agencies  utilize  intermediate  panels,  such  as  the 
SSAC  in  DOD  major  acquisitions,  which  exercise  independent 
judgment  in  reviewing  the  findings  of  the  lower  evalution 
panels.  Such  procedures  have  also  been  recognized  as  being 
within  the  discretion  of  contracting  agencies  by  the 
Comptroller  General.82 

Not  surprisingly,  there  is  broad  agency  discretion  in 
determining  the  composition  of  evaluation  boards.83 
Obviously,  procuring  agencies  attempt  to  utilize  the  most 
capable  evaluators  available  and  the  Comptroller  General  will 
not  normally  become  involved  in  appraising  the  qualifications 
of  evaluators.84  Further,  internal  agency  guidelines 
concerning  the  composition  of  evaluation  boards  do  not  create 
substantive  rights  in  offerors;  therefore,  failure  to  follow 
such  guidelines  provides  no  basis  for  objection.85 

2 .  Bias  and  Conflicts  of  Interest 


The  Comptroller  General  will  question  agency 
discretion  in  the  area  of  evaluation  personnel  if  there  is  a 
showing  of  bias  or  conflict  of  interest.  As  to  the  former. 
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the  protestor  has  the  burden  of  proof  in  demonstrating  an 
evaluator  has  acted  in  a  biased  manner.86  In  a  protest 
against  the  award  of  a  contract  by  HEW,  the  Comptroller 
General  stated: 

In  regard  to  what  is  necessary  to  make  a  showing 
of  bias,  we  have  stated  that  'It  must  be 
emphasised  *  *  *  that  unfair  or  prejudicial 
motives  will  not  be  attributed  to  individuals  on 
the  basis  of  inference  or  supposition.'  A.R.F. 
Products,  Inc.,  56  Comp.  Gen.  201,  208  (1976), 

76-2  CPD  541. 8' 

As  to  conflicts  of  interest,  all  officers  and 
employees  of  the  Federal  Government  are  bound  by  certain 
standards  of  conduct  which  prohibit  them  from  placing 
themselves  in  a  position  of  conflict,  by  virture  of  their 
official  duties,  between  their  private  interests  and  the 
public  interests  of  the  United  States.88  Agency  evaluation 
personnel  find  themselves  in  an  inherently  sensitive  position. 
Therefore,  many  regulations  require  such  personnel  to  be 
specifically  briefed  on  their  duties  and  responsibilities  with 
regard  to  real  or  apparent  conflicts  of  interest.89  while 
the  Comptroller  General  has  acknowledged  that  the 
responsibility  for  determining  whether  a  conflict  of  interest 
exists  and  for  enforcing  requisite  standards  of  conduct  rests 
with  the  procuring  agency,  he  has  on  occasion  commented  upon 
alleged  violation  of  standards  as  they  relate  to  particular 
procurements.90  For  example,  he  has  stated  that  it  is 
incumbent  on  the  agency  to  avoid  even  the  appearance  of 
favoritism  or  preferential  treatment.91 
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As  such,  where  one  of  four  evaluators  disqualified  herself 
from  evaluating  a  specific  proposal  because  of  a  potential 
conflict  of  interest,  the  agency  should  have  disqualified  her 
from  evaluating  the  remaining  proposals.92  In  another  case 
where  an  agency  learned  the  Contracting  Officer's  daughter  was 
employed  by  the  ultimate  awardee,  it  would  have  been 
preferable  to  reassign  the  Contracting  Officer.93  However, 
no  relief  was  forthcoming  in  that  case  since  the  Contracting 
Officer  denied  any  wrongdoing,  and  brought  the  matter  to  the 
attention  of  her  superiors  (who  then  determined  no  conflict  of 
interest  existed). 

Even  though  agencies  should  avoid  even  the  appearance 
of  a  conflict  of  interest,  the  mere  presence  of  a 
"possibility"  thereof  is  insufficient  to  support  a  protest.94 
Simple  allegations  are  likewise  insufficient  as  the 
Comptroller  General  will  not  conduct  investigations  for  the 
purpose  of  establishing  the  validity  of  unsubstantiated 
statements.95  Thus,  the  protestor  must  present  sufficient 
evidence  to  affirmatively  prove  his  or  her  allegations.96 

E.  Methods  of  Evaluation 

1 .  General  Considerations 

In  the  beginning  of  this  chapter  it  was  stated  that 
one  common  thread  seems  to  bind  the  source  selection  process 
together  --  agency  discretion.  Just  as  procuring  activities 
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exercise  great  discretion  in  selecting  and  defining  their 
minimum  needs,  and  in  shaping  the  structure  of  evaluation 
procedures,  so  do  they  enjoy  such  discretion  in  the  choice  of 
methods  and  techniques  to  be  used  in  the  actual  evaluation  of 
proposals . 97 

Generally,  agency  determinations  during  the  evaluation 
process  are  entitled  to  great  weight.98  The  Comptroller 
General  has  repeatedly  stated  that  it  is  not  his  function  to 
evaluate  proposals  to  determine  their  eligibility  for 
award.99  The  judgments  of  the  procuring  activities’ 
technicians  and  specialists  as  to  the  adequacy  of  proposals 
will  be  accepted  absent  a  clear  showing  of  unreasonable¬ 
ness.100  Further,  agencies  may  use  their  own  independent 
estimates  as  evaluation  aide.101  These  estimates  can  pertain 
to  costs,102  and  to  items  which  contribute  to  costs,  such  as 
required  manhours.103 

The  mere  fact  that  a  protestor  disagrees  with  an 
agency's  evaluation  does  not  render  the  evaluation 
unreasonable.104  Where  there  is  a  difference  in  judgment, 
the  procuring  agency's  good  faith  judgment  must  prevail.105 
An  excellent  and  succinct  comment  by  the  Comptroller  General 
concerning  agency  discretion  in  this  area  can  be  found  in  a 
1980  protest  by  Pacific  Consultants,  Inc.108  In  it  he 
stated: 


Before  discussing  the  specific  complaints  raised 
by  Pacific,  we  note  that  in  resolving  cases  in 
which  a  protestor,  as  here,  challenges  the 
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validity  of  a  technical  evaluation,  it  ie  not 
the  function  of  this  Office  to  evaluate 
proposals  in  order  to  determine  which  should 
have  been  selected  for  award.  The  determination 
of  the  relative  merits  of  proposals  is  the 
responsibility  of  the  procuring  agency  since  it 
must  bear  the  burden  of  any  difficulties 
incurred  by  reason  of  a  defective  evaluation. 

In  light  of  this,  we  have  held  that  procuring 
officials  enjoy  a  reasonable  degree  of 
discretion  in  the  evaluation  of  proposals  and 
such  discretion  must  not  be  disturbed  unless 
shown  to  be  arbitrary  or  in  violation  of  the 
procurement  statutes  and  regulations.  *  *  * 

Thus,  our  Office  will  not  substitute  its 
judgment  for  that  of  the  procuring  agency  by 
making  an  independent  determination.  *  *  * 
Additionally,  the  protestor  has  the  burden  of 
affirmatively  proving  its  case.107 


The  only  restrictions  on  procuring  agencies  in 
determining  the  particular  method  of  evaluation  are  that  the 
method  have  a  rational  basis  and  that  the  evaluation  be 
conducted  in  a  good  faith  manner  and  in  accordance  with 
disclosed  evaluation  criteria.108  These  restrictions  are 
further  discussed  in  succeeding  chapters  of  this  review. 


2 .  Common  Techniques 

Procuring  agencies  have  tried  many  techniques  in  an 
attempt  to  represent  the  essentially  subjective  decisions  of 
those  individuals  involved  in  the  evaluation  process. 

Agencies  have  used  adjectives,  narratives,  symbols,  color 
codes  and  numerical  formulae  in  their  evaluations.  Normally, 
all  evaluation  systems  have  some  form  of  narrative  complement 
to  the  basic  system  which  ties  the  process  together.  The  use 
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of  adjectives  such  as,  "outstanding,  excellent,  good,  fair, 
poor"  and  the  use  of  color  codes  usually  involves  the 
placement  of  proposals  and  their  specific  aspects  in  certain 
ranges,  which  are  represented  by  each  adjective  or  color. 

This  presumably  allows  more  trade-off  flexibility  within  those 
given  ranges  than  would  precise  numerical  scores. 

a .  Adjectival /Narrative 


Numerical  formulae  are  firmly  embedded  agency 
evaluation  techniques,  even  though  they  are  not  required. 109 
Yet,  subjective  rating  systems  have  been  the  subject  of  far 
less  protests.  Perhaps  this  fact  should  provide  at  least  food 
for  thought  amongst  the  agencies.  In  one  of  the  few  protests 
concerning  a  purely  subjective  system,  NASA's  use  of  an 
adjectival  rating  system  was  challenged.11®  Quite  simply, 
the  Comptroller  General  stated  that  his  Office  had  no 
objection  to  the  use  of  such  a  system,  as  long  as  it  met  the 
usual  test  of  rationality.  The  protestor  had  argued  the 
adjective1  system  masked  the  decision-making  process,  and,  as 
such,  offerors  were  not  provided  the  rationale  employed  in 
making  the  decision.  In  another  demonstration  of 
discretionary  latitude,  the  Comptroller  General  sanctioned  a 
procuring  agency's  abandonment  of  a  point-scoring  scheme  and 
substitution  of  a  simple  ranking  system.111  The  proposals 
were  point-scored  initially,  then  each  was  ranked 
accordingly.  All  were  discussed  by  the  evaluation  panel.  The 
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protestor  claimed  that  such  a  system  magnified  small 
differences  in  scores  (e.g.,  the  first,  second  and  third 
ranked  proposals  scored  95,  94  and  93,  respectively)  and 
distorted  the  fact  that  some  were  essentially  equal. 
Reiterating  his  usual  stand  that  point  scores  are  only  useful 
guides,  the  Comptroller  General  said  it  was  not  unreasonable 
to  expect  that  in  a  given  case  ranking  may  be  a  more  direct 
and  meaningful  method  if  it  allows  procurement  officials  to 
gain  a  clearer  understanding  of  the  merits  of  competing 
proposals . 

The  regulations  surveyed  mentioned  adjectival/nar¬ 
rative  techniques  with  varying  degrees  of  specificity.  Of 
those  regulations,  the  Air  Force  provides  the  most  detailed 
material.  afr  70-15  cites  two  common  methods  for  use  in 
scoring  below  the  Item  level.  They  are  the  "numerical"  method 
and  the  "narrative"  method.112  As  the  name  suggests,  the 
numerical  method  involves  the  assignment  of  a  pre-established, 
prorated  numerical  scale  designated  to  the  specific  Factors. 
Caution  i 8  suggested  in  the  use  of  numerical  scores,  and 
cost/price  is  never  to  be  numerically  scored.113  The 
regulation  points  out  that  the  narrative  method  has  been  found 
to  be  a  better  approach  at  this  level  of  evaluation.114  In 
that  regard,  a  symbol  or  color  code  may  be  used  as  an  aid  in 
determining  how  well  the  proposal  met  the  standard  for  each 
Factor.  For  example,  "+*,  *  ",  and  "-*  signs  can  be  used  to 
show  that  an  offer  has  exceedsd,  met  or  failed  to  meet  minimum 
requirements.115  When  a  color  cods  scheme  is 
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used,  the  following  spectrum  must  be  utilized: 

Exceptional,  or  exceeds  standards  -  blue 
Meets  standards  -  green 

Marginal,  or  fails  to  meet  standards  -  yellow 
Unacceptable  -  red116 

when  the  rectors  have  been  evaluated,  the  Item  designator  is 

determined  from  the  following  scale: 

10  -  Exceptional  Blue 

9 

-  Exceeds  Standards 

8 

7 

6  -  Meets  Standards  Green 

5 
4 
3 

-  Fails  to  Meet  Standards  Yellow 

2 

1 

0  -  Unacceptable  Red117 


Specific  guidance  in  the  application  of  this  scale  is 
provided.  For  example,  if  the  majority  of  the  Factors  meet 
standards,  the  requirement  is  not  overly  difficult,  and  the 
deficient  factors  are  minor  in  nature,  the  Item  should  be 
scored  "5". 118  Scores  of  "6"  or  "7"  should  normally  be 
assigned  to  "relatively  difficult"  requirements  where  the 
majority  of  the  Factors  are  acceptable,  with  no  major 
deficiencies  or  risks  existing  therein,  and  the  collectve 
approach  yields  a  qualitative  benefit  to  the  Government . 1 19 
However,  where  most  of  the  major  Factors  are  acceptable,  but 
one  or  more  are  deficient  and  involve  some  minor  risk,  the 
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score  "f or  the  time"  should  be  "4". 120  A  *3,  3  or  1"  is 
assigned  where  a  majority  of  the  Factors  are  deficient  and 
correction  involves  problems  or  has  a  "domino  effect"  on  other 
features.121  The  latter  scores  are  also  assigned  to  those 
approaches  which  contain  high  risks , 122  without  means  for 
correction  if  they  fail.  When  a  "major  reorientation"  of  the 
proposal  is  necessary  the  Item  is  scored  "0".123  The  highest 
numbers,  "8,  9  and  10",  are  assigned  to  difficult  requirements 
which  are  met  with  approaches  that,  with  little  risk,  will 
yield  results  which  qualitatively  exceed  such 
requirements.124  Further,  with  regard  to  scores  above  "5", 
the  regulation  provides  that  such  scores  must  not  be  assigned 
"simply  because  a  proposal  offers  to  exceed  a  requirement  in 
some  quantitative  way."125  That  is,  the  offer  may  not 
necessarily  be  an  advantage  if  it  will  not  add  substantially 
to  the  contemplated  mission.  Obviously,  the  assignment  of  a 
score  above  "5"  should  reflect  some  qualitative  achievement. 

b.  numerical  Formulae 


With  regard  to  numerical  systems,  the  Comptroller 
General  has  recognized  their  relative  usefulness,  stating  that 
such  ratings  are  useful  guides  in  the  evaluation  process,  but 
are  not  conclusive  as  to  the  actual  adequacy  of  individual 
proposals.126  They  are  used  in  an  "attempt  to  quantify  what 
is  essentially  a  subjective  judgment,127  and  "only  reflect 
the  disparate  judgments  of  the  evaluators  and  thus  a 
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difference  in  scores  nay  not  reflect  an  actual  difference  in 
merit."128  Numerical  point  ratings  are  sometimes  used  to 
score  only  initial  offers,  while  best  and  final  offers  are 
subjectively  analyzed,128  or  both  initial  and  best  and  final 
offers  may  be  numerically  scored. 110  The  scores  assigned  by 
evaluators  are  most  often  utilized  in  proposal  evaluation,  but 
in  certain  instances  suc.'i  scoring  is  "normalized",  so  that  the 
highest  rated  proposal  is  equated  to  a  maximum  score.131 
Again,  agency  discretion  predominates,  as  systems  can  be  broad 
or  very  detailed. 

All  of  the  examined  regulations  mentioned 
numerical  scoring,  again  with  differing  levels  of  detail .  DOE 
simply  notes  that  two  principal  techniques  of  rating  proposals 
are  numerical  and  adjectival,  each  having  the  same  meaning  as 
discussed  in  regard  to  the  military  systems.132  Technical 
evaluation  criteria  are  point  scored  and  assigned  numerical 
weights,  while  business/management  criteria  are  usually  rated 
by  the  adjectival  method. 133  Cost  is  not  assigned  a 
numerical  weight,  or  an  adjectival  rating,  but  is  "used  in  the 
ultimate  decision  by  the  Source  Selection  Official  to  judge 
the  value  of  the  work  to  be  done  and  the  quality  of  the 
product  or  service  to  be  furnished,  and  not  as  an  addition  to 
the  cumulative  score  of  the  other  analysis.”134 

A  DOE  point  scoring  system  was  challenged  in  a  1980 
protest  because  it  allegedly  tended  to  enlarge  any  difference 
in  technical  merit  and  distort  the  relative  importance  of  that 
difference. 135  The  system  used  a  scale  of  *  1 0-8-5-2-0" .  The 
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Comptroller  General  stated  that  while  a  more  refined  method 
(e.g.,  "1-3-4-6-7-9")  may  have  reduced  any  unintended 
distortions,  it  was  ultimately  for  the  source  selection 
official  to  determine  any  significance  in  such  scores,  and  it 
could  not  be  said  that  the  decision  to  award  was  not 
rationally  founded.  Thus,  once  again  discretion  prevailed. 

NASA  uses  an  undefined  numerical  scoring  system  for 

Mission  Suitability  as  noted,  with  two  added  cautions: 

(1)  It  is  only  as  good  as  the  judgment  made 
in  selecting  criteria  and  weights. 


(2)  If  care  is  not  exercised  to  limit 
reasonably  the  number  of  subdivisions  to  be 
rated,  the  scoring  system  will  introduce  an 
"averaging  out*  effect  that  inhibits  selection 


based  on  the  really  significant 
among  the  offerors.136 


discriminators 


Predetermined  weights  are  assigned,  following  evaluation, 
normally  utilizing  1000  points  as  a  perfect  score.137 
Specific  weightings  are  not  divulged  to  evaluators  below  the 
level  of  the  SEB  itself,  i.e.,  to  the  committee  level-138 

Perhaps  it  should  be  noted  here  that  when  numerical 
scoring  is  used,  it  is  necessary  to  weigh  the  respective 
criteria  in  order  to  reflect  their  appropriate  relative 
importance.  A  simple  method  is  weight  assignment  by 
percentage  distribution.139  Another  common  method  of 
weighting  can  be  found  in  APR  70-15,  which  provides  for  the 
use  of  weights  by  the  SSAC  whan  numerical  scoring  is 
utilized.140  They  are  consistent  with  the  requisite 


solicitation  disclosures  and  are  established  prior  to  the 
receipt  of  proposals.  Weights  in  the  Air  Force  cannot  be 
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divulged  to  the  SSEB,  nor,  in  accordance  with  OAR,  to 
potential  offerors. Under  the  regulation,  weighting  may 
be  accomplished  by  assigning  each  scorable  Item  of  the 
evaluation  criteria  a  share  of  1000  points  so  that  the  end 
result  represents  the  relative  importance  accorded  each  Item 
in  the  evaluation  process.  In  order  to  do  this,  the  numerical 
score  assigned  by  the  SSEB  (raw  acore)  is  multiplied  by  the 
weighting  factor  developed  by  the  SSAC  and  the  result  is 
divided  by  10,  the  maximum  raw  score  achievable.  The  formula 
appears  as  follows: 

(Raw  score  x  Weighted  factor  -  Weighted  score)142 

10 

Before  leaving  the  Air  Force  at  this  point,  it  might 
be  mentioned  that  AFR  70-15  is  presently  undergoing  revision. 
Mo  estimate  on  a  publication  date  for  the  new  regulation  is 
possible.  However,  while  no  basic  changes  in  policy  or 
procedures  are  contemplated,  it  is  likely  the  Air  Force  will 
encourage  the  use  of  color  coding  techniques  rather  than 
numerical  scoring,  at  least  in  major  systems  acquisitions.143 
Seemingly,  the  feeling  is  that  the  use  of  color  coding 
provides  a  greater  margin  for  discretion  and  avoids  the 
inelastic  connotations  of  numerical  scores.  Thus,  subjective 
evaluations  of  strengths,  weaknesses  and  risks  of  proposals 
falling  within  a  particular  color  code  may  presumably  be 
accomplished  with  more  flexibility. 
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C.  Combined  Methods 


In  many  major  systems  acquisitions,  adjectival/nar¬ 
rative  methods  of  evaluation  are  frequently  used  in 
conjunction  with  numeric  systems*  The  agency  regulations 
specifically  provide  for  such  combinations.  Indeed  the  Air 
Force  color  code  cited  above  incorporates  numerical  values. 

In  the  Army,  DARCOM  P  715-3  states  that  the  methods 
used  for  evaluating  proposals  should  focus  on  "realizing  the 
highest  attainable  measure  of  objectivity."144  The  "core"  of 
the  evaluation  process,  states  the  Army,  is  scoring. 145 
Scoring  may  be  accomplished  by  subordinate  commands  via 
several  methods,  no  particular  method  being  preferred.  The 
"numeric  method"  is  cited  as  the  most  frequently  used, 
scores  in  such  method  being  normally  based  on  a  preestablished 
scale  from  zero  to  ten.146  Each  numerical  score  must  be 
accompanied  by  a  supporting  narrative  signed  by  the  evaluator 
which  discusses  strong  and  weak  points.147  Adjectival 
scoring  is  noted  as  often  being  employed  in  connection  with 
numerical  ratings.148  A  typical  Army  numerical-adjectival 
combined  matrix  is  shown  below: 

SCORE  EVALUATION 

10  Excellent  --  (innovative,  comprehensive  and 

complete  in  all  details,  meets  all  require¬ 
ments  and  objectives  without  gold  plating). 
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9  Very  Good  --  (substantial  response  in  clearly 

definable  detail.  Beets  all  critical  require¬ 
ments)  . 

7  Average  --  (generally  meets  minimum 

requirements ) . 

6  Poor  --  (lack  of  essential  information  to 

substantiate  data  presented). 

5  Unsatisfactory  --  (lack  of  understanding  of 

requirements  or  omissions  in  major  areas). 

0  No  data . 

In  the  Navy,  it  is  necessary  to  refer  to  one  of  NAVMAT's 
subordinate  commands  in  order  to  find  guidance  on  techniques. 
NAVELEX  Instruction  4200. 12C  requires  that  "a  numerical 
scoring  system  will  be  employed  which  will  translate  the  word 
descriptions  into  quantitative  terms."150  Briefly,  the 
■ubstance  of  that  system  is  as  follows: 

(1)  "Score  81  to  100%  -  Excellent."  Offeror 
displays  highest  levels  of  innovation,  technical 
competence  and  managerial  ability,  and  fully 
meets  expectations. 

(2)  "Score  61  to  80%  -  Good."  Offeror 
demonstrates  acute  awareness  of  requirements, 
and  his  technical  and  planning  efforts  show 
strong  promise. 

(3)  "Score  41  to  60%  -  Fair."  Offeror  presents 
technically  correct,  orderly  plan  to  meet 
requirements,  but  demonstrates  no  exceptional 


features 
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(4)  "Score  21  to  40%  -  Poor."  Offeror 
indicates  less  than  full  understanding  of 
requirements,  and  fails  to  demonstrate  a 
reasonable  probability  of  performing  the 
desired  tasks,  or  his  approach  is  risky,  even 
though  his  analysis  is  technically  correct. 

(5)  "Score  0  to  20%  -  Unacceptable."  Does  not 
meet  the  requirements  stated  in  the  RFP. 

(Note:  Xn  unacceptable  rating  in  one  or  more 

"critical"  areas  may  make  the  entire  proposal 
unacceptable,  even  though  the  total  score  may  be 
considerably  higher  than  20%.)^1 

3 .  Cost/Price  Evaluations  -  Special  Problems 

a .  General  Considerations 


Like  other  evaluation  criteria,  agencies  must 
determine  what  role  cost/price  will  play  in  a  particular 
procurement,  as  well  as  how  that  role  will  be  evaluated.  X 
third  consideration  is  how  much  of  the  above  two 
determinations  should  be  disclosed  to  industry.  We  have 
already  discussed  cost/price  as  an  evaluation  criterion. 
Disclosure  considerations  will  be  discussed  in  the  following 
chapter.  This  section  will  examine  various  cost/prica 
evaluation  methods  and  their  Inherent  difficulties. 
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The  evaluation  of  cost/price  presents  special 
problems/  and  agencies  have  used  a  variety  of  methods  to 
combat  them.  As  noted,  some  procuring  agencies  consider  costs 
or  prices  without  the  use  of  numerical  scores.  When  so 
considered,  costs  may  be  compared  to  other  factors,  ever 
though  the  latter  are  numerically  point-scored.152 
Cost/price  may  also  be  evaluated  numerically  and  the  resultant 
score  totalled  with  other  evaluation  factor  scores.155  When 
the  latter  method  is  used,  costs  are  usually  scored  by 
awarding  the  maximum  amount  of  allowable  points  to  the  lowest 
priced  technically  acceptable  offer,  with  proportionately 
lesser  points  given  to  higher  priced  offers.154  For  example, 
one  agency  weighted  technical  proposals  at  80  percent,  with  a 
corresponding  maximum  point  value  of  80  points.155  Price 
proposals  were  weighted  at  20  percent,  with  a  20  point 
maximum.  Price  proposal  scores  were  computed  by  assigning  20 
points  to  the  lowest  proposal,  dividing  each  of  the  other 
prices  into  the  lowest  price,  and  then  multiplying  the  result 
by  20.  In  that  regard,  it  might  be  worthwhile  to  note  here 
that  when  a  solicitation  sets  forth  a  precise  numerical 
evaluation  formula  and  provides  award  will  be  made  on  the 
basis  of  the  total  point  score,  then  the  highest  scored 
acceptable  proposal  should  be  selected,  i.e.,  trade-offs  are 
limited. 156 

The  Comptroller  General  has  even  sanctioned  the 
practice  of  quantifying  technical  point:  scores  in  terms  of 
relative  dollar  advantage  by  computing  cost/quality 
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ratios.  The  Army  specifically  provides  for  the  use  of  such 
ratios,  classifying  them  as  "special  factors"  to  be  considered 
in  appropriate  procurements.158  This  ratio  is  established  by 
dividing  the  cost  of  the  proposal  by  the  total  unweighted  raw 
score  developed  for  technical  considerations.  The  Army  notes 
that  the  ratio  by  itself  is  not  justification  for  selection, 
but  is  only  an  additional  factor  to  be  considered.  However, 
they  believe  that  the  offerors'  knowledge  that  this 
relationship  will  be  given  consideration  will  force  them  to 
trade  off  between  cost  and  technical  factors  "in  order  to 
prepare  the  best  possible  proposal  at  a  fair  and  reasonable 
price."159  While  the  Navy's  regulations  that  were  surveyed 
do  not  specifically  mention  "$/q.p."  ratios,  they  have  been 
criticized  for  utilizing  them  as  the  primary  reason  for 
awarding  "turnkey"  housing  contracts  on  initial  proposals.160 
"Turnkey"  procurements  for  family  housing  usually  allow  widely 
varied  approaches  and  designs,  within  the  constraints  of 

performance  specifications.  As  such  the  Comptroller  General 

\ 

has  stated  that: 

l 

Because  of  this  wide  variance  of  approaches, 
although  an  offeror  has  received  the  lowest 
$/q.p.  ratio,  that  does  not  mean  it  is 
necessarily  offering  such  a  "fair  and 
resonable"  price  that  oral  or  written 
discussions  would  not  be  required,' 
notwithstanding  the  existence  of  several 
competitive  offerors,  since  a  true  basis  for 
comparison  of  the  proposals  to  insure  a  "fair 
and  reasonable"  price  was  received  may  be 
lacking. 161 

Zn  any  case,  the  evaluation  of  cost  or  price  via 
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numerical  formulae  can  be  precarious.  Indeed,  some 

procurement  regulations  emphasize  that  such  methods  are  only 

to  be  considered  rough  yardsticks.  The  Comptroller  General 

has  also  found  it  necessary  on  many  occasions  to  criticize 

agency  numerical  scoring  methods  of  cost  or  price.  Thus,  it 

is  inappropriate  to  point-score  proposals  on  price  when  those 

proposals  have  no  reasonable  chance  of  being  selected  for 

award,  particularly  when  those  proposals  are  also 

significantly  lower  priced  than  those  in  the  competitive 

range. In  a  1980  protest,  the  Comptroller  General  stated: 

We  do  not  understand  what  purpose  is  served 
by  point-scoring  proposals  on  price  when 
those  proposals  have  no  reasonable  chance  of 
otherwise  being  accepted,  and,  as  indicated 
in  Francis  &  Jackson,  when  such  proposals  are 
also  significantly  lower  priced  than  those 
which  are  in  the  range  of  acceptability, 
their  inclusion  in  the  price  scoring  could 
distort  the  evaluation  results.163 

Nevertheless,  the  test  of  rationality  prevailed  once  again, 
with  the  Comptroller  General  finding  that  the  selection  was 
not  irrational  or  otherwise  illegal.  However,  he  cautioned 
source  selection  officials  to  be  aware  of  the  possible 
misleading  results  of  such  evaluation  techniques  and  to  make 
certain  that  awards  are  not  made  "automatically  on  the  basis 
of  the  results  of  point  scoring."164 

Similarly,  the  inclusion  of  a  very  high  price  can 
result  in  an  improper  "bunching"  of  scores  for  other,  more 
realistic  prices.  Specifically,  "bunching"  occurred  where  a 
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procurli 'v  agency  assigned  maximum  points  to  any  coat  proposal 
falling  within  a  range  of  $25,000.00  to  $60 , 000 . 00 . The 
Comptroller  General  opined  that  such  an  evaluation  could  lead 
to  distorted  results  where  award  might  be  made  to  an  offeror 
at  a  much  higher  price  simply  because  such  offeror  has  a 
slight  technical  advantage.  Further,  such  an  evaluation  has 
the  effect  of  eliminating  cost  as  an  evaluation  factor. 

Likewise,  a  formula  which  assigns  "essentially" 
equal  scores  to  all  proposals  can  have  the  same  detrimental 
effects.  Thus,  cost  scoring  based  upon  the  following  formula 
proved  unacceptable: 

166 

Average  Cost  of  All  Technically  Acceptable  Proposalsx20«CoBt 
Offeror's  Proposed  Cost  Score 

Obviously,  since  no  offeror  could  be  assigned  more  than  20 
points  for  cost,  all  cost  proposals  below  the  average  cost 
received  the  maximum  20  points.  Those  which  proposed  more 
than  the  average  cost  received  slightly  lower  scores.  The 
Comptroller  General  found  that  the  application  of  this 
formula  resulted  in  the  majority  of  initially  acceptable 
offerors  receiving  the  maximum  number  of  points.  Also,  there 
was  no  meaningful  difference  between  scores  assigned  offerors 
even  though  the  proposed  costs  ranged  from  $10,810.00  (20 
points)  to  $23,216.00  (18  points). 


49 


Another  improper  evaluation  was  found  in  a  1981 
protest  when  ©rices  proposed  by  13  original  offerors  were 
included  in  a  final  cost  evaluation  of  the  four  offerors 
within  the  competitive  range.167  This  action  resulted  in 
cost  receiving  less  than  the  announced  40%  weight. 

One  simple  lesson  can  be  learned  from  the  numerous 
cases  on  the  numerical  scoring  of  cost/price:  agencies  must 
be  careful  and  ensure  that  any  such  scoring  does  not  include 
the  evaluation  of  offers  which  would  cause  improper 
distortions.  Yet,  in  spite  of  what  appears  to  be  a  clear 
lesson,  protests  and  criticisms  of  agency  methods  continue. 
No  doubt  the  frequent  ultimate  denial  of  such  protests  based 
upon  grounds  of  rationality  or  lack  of  prejudice  contributes 
to  agency  complacency.168 

b.  Cost/Price  Analysis 

The  core  of  cost/price  evaluation  is  an  agency's 
analysis  of  the  proposed  cost/price  in  each  case.  The  DAR, 
PPR  and  NASA  PR  all  require  that  some  form  of  either  cost  or 
price  analysis  be  utilized  in  all  negotiated  procurements, 
the  method  and  degree  of  analysis  being  dependent  upon  the 
particular  nature  of  the  procurement  and  the  pricing 
situation  involved.169  Cost  analysis  must  be  performed 
whenever  cost  or  pricing  data  is  required  to  be  submitted  by 
offerors,  while  price  analysis  must  be  performed  whenever 


so 


cost  analysis  is  not,  and  it  should  be  used  to  supplement 
cost  analysis  whenever  appropriate.170  Further,  specific 
methods  of  performing  price  and  cost  analysis  are  provided 
by  the  procurement  regulations.171 

In  addition  to  the  DAR  provisions,  the  Air  Force 
emphasizes  that  cost/price  evaluations  should  consist  of  an 
assurance  of  comparability,  a  verification  of  rates,  a 
determination  of  cost/price  realism,  and  a  special 
assessment  for  any  cost/price  which  appears  to  be 
unrealistic.172  The  principal  aims  of  NASA  cost  analyses 
are  validity  of  proposed  costs,  the  probable  cost  to  the 
Government  of  each  proposal,  the  probable  cost  differences 
among  the  proposers  and  their  causes,  and  the  level  of 
confidence  in  their  cost  analyses.173 

Generally,  an  agency’s  judgment  as  to  the 
methods  necessary  to  perform  an  adequate  analysis  is 
entitled  to  great  weight.174  In  the  1976  protest  of  Grey 
Advertising,  Inc.,  the  Comptroller  General  stated  that  he 
"will  not  second-guess  a  cost-realism  determination  unless 
it  is  not  supported  by  a  reasonable  basis."175  Even  though 
the  Navy  conducted  a  somewhat  less  than  "in-depth"  cost 
analysis,  their  determination  was  nevertheless  reasonable. 
There,  no  independent  verification  or  otherwise  in-depth 
analysis  of  proposed  costs  was  undertaken  in  an  indefinite 
quantity  contract.  The  Comptroller  General  found  that  where 
an  agency  can't  precisely  Identify  future  requirements  and 
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Bust  request  estimated  costs  on  the  besis  of  e  hypothetical 

plan,  such  estimated  costs  can  provide  an  adequate  basis  for 

cost  comparison  between  competing  proposals* 

Although  some  form  of  analysis  is  required  in 

all  negotiated  contracts,  most  of  the  controversy 

surrounding  realistic  cost  analysis  is  found  in  the  area  of 

cost  reimbursement  contracts.  As  to  the  role  of  cost  in 

such  contracts,  OAR  3-803(e)  states: 

In  selecting  the  contractor  for  a 
cost-reimbursement  type  contract,  estimated 
costs  of  contract  performance  and  proposed 
fees  should  not  be  considered  as 
controlling,  since  in  this  type  of  contract 
advance  estimates  of  cost  may  not  provide 
valid  indicators  of  final  actual  costs. 

There  is  no  requirement  that 
cost-reimbursement  type  contracts  be 
awarded  on  the  basis  of  either  (i)  the 
lowest  proposed  cost,  (ii)  the  lowest 
proposed  fee,  or  (iii)  the  lowest  total 
estimated  cost  plus  proposed  fee.  The 
award  of  cost-reimbursement  type  contracts 
primarily  on  the  basis  of  estimated  costs 
may  encourage  the  submission  of 
unrealistically  low  estimates  and  increase 
the  likelihood  of  cost  overruns.  The  cost 
estimate  is  Important  to  determine  the 
prospective  contractor's  understanding  of 
the  project  and  ability  to  organize  and 
perform  the  contract.  The  agreed  fee  must 
be  within  the  limits  prescribed  by  law  and 
appropriate  to  the  work  to  be  performed 
(see  3-808). ,  Beyond  this,  however,  the 
primary  consideration  in  determining  to 
whom  the  award  shall  be  made  is:  which 
offeror  can  perform  the  contract  in  a 
manner  most  advantageous  to  the  Government. 


FPR  1-3. 8051(d)  and  RASA  PR  3.80-2  contain  similar 


provisions 
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While  the  methods  employed  in  obtaining  a  proper 
cost  analysis  may  be  discretionary,  at  a  minimum  some  form 
of  independent  determination  of  cost  realism  is  required  in 
cost  reimbursement  contracts.176  The  Comptroller  General 
has  stated: 

The  award  of  cost  reimbursement  contracts 
requires  the  exercise  of  informed  judgments 
as  to  whether  proposed  costs  are  realistic, 
and  it  is  improper  to  award  such  a  contract 
on  the  basis  that  such  costs  are  reasonable 
because  they  are  low  per  se  on  a 
comparative  basis,  if  the  Government  fails 
to  adequately  measure  the  realism  of  such 
low  costs • 1 7  7 

Thus,  it  is  improper  to  take  proposed  coats  at  face  value 
simply  because  an  awarder  has  a  "past  history  of 
frugality."178  Likewise,  inadequate  analyses  include  a 
simple  comparison  of  proposed  costs,179  ignoring 
unrealistic  cost  proposals  by  assigning  equal  scores  to  all 
cost  proposals,180  and  simply  assigning  the  highest  point 
score  to  the  lowest  proposed  cost.181 

What  these  requirements  mean  is  that  when  a  cost 
reimbursement  contract  is  involved,  evaluated  costs  rather 
than  proposed  costs  provide  a  more  sound  basis  for 
determining  the  proposal  most  advantageous  to  the 
Government,  since  the  Government  is  required,  within  certain 
limits,  to  pay  a  contractor's  actual,  allowable  and 
allocable  costs  in  such  contracts.182  while  the 
Comptroller  General  has  agreed  with  regulatory  requirements 
for  independent  Government  cost  estimates,  he  has  also 
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cautioned  against  undue  reliance  on  then,  in  view  of  the 
inherent  performance  uncertainties  in  cost  reimbursement 
contracts . 183 


C.  normalization  Techniques 


The  determination  of  cost  realism  in  proposals 
is  the  substance  of  cost  analysis.  In  creating  systems 
designed  to  determine  such  realism,  agencies  sometimes  use  a 
technique  commonly  called  "normalization".  Like  other 
methods  of  analysis,  normalization  should  also  be  used  only 
when  appropriate  to  the  particular  procurement. 

Normalization  has  been  described  by  the 
Comptroller  General  in  the  1975  protest  of  Dynalectron 
Corporation  as s 


Normalization  is  a  technique  sometimes 
used  within  the  cost  adjustment  process  in 
an  attempt  to  arrive  at  a  greater  degree  of 
cost  realism.  It  involves  the  measurement 
of  at  least  two  offerors  against  the  same 
cost  standard  of  baseline  in  circumstances 
where  there  is  no  logical  basis  for 
differences  in  approach,  or  in  situations 
where  insufficient  information  is  provided 
with  the  proposals,  leading  to  the 
establishment  of  common  "should  have  bid" 
estimates  by  the  agency.  See  Hatter  of 
Lockheed  Propulsion  Company.  S- 173677,  June 
24,  1974,  5  3  Comp.  Gen.™4 


As  such,  where  the  requirements  of  the  contract  call  for 
flexibility  or  individualized  approaches,  normalization  is 
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improper.  Examples  of  good  and  bad  normalizations  mere 
present  in  Dynalectron  Corporation.  An  appropriate 
normalization  concerned  the  normalized  coats  proposed  by 
offerors  for  paymnent  of  a  state  tax,  since  such  tax  was 
applicable  to  all  offerors,  whereas  normalization  of  labor 
costs  involved  in  individualized  approaches  for  certain 
services  were  inappropriate.  In  that  regard,  the 
Comptroller  General  stated: 


The  proper  goal  in  both  instructing 
offerors  as  to  proposal  preparation  and  in 
conducting  the  probable  cost  evaluation 
itself  is  to  segregate  cost  factors  which 
are  "company  unique "--dependent  on 
variables  resulting  from  dissimilar  company 
policies--f rom  those  which  are  generally 
applicable  -to  all  offerors  and  therefore 
subject  to  normalization. 185 


Thus,  if  the  subject  of  the  normalization 
technique  fits  the  commonality  requirement,  the  Comptroller 
General  won't  object  unless  the  analysis  lacks  a  reasonable 
basis.186  However,  unreasonableness  can  be  found  in 
neglecting  to  normalize  when  appropriate.  In  one  protest  it 
was  found  unreasonable  for  an  agency  to  fail  to  normalize 
cost  differences  when  a  common  cost  per  pound  for  a  certain 
item  was  clearly  apparent.187  In  another  case,  failure  to 
normalize  proposed  airline  fare  costs  was  improper.188 

The  term  "normalized"  is  also  applied  to  other 
techniques  whereby  cost  is  converted  into  certain  ratios 
involving  technical  considerations,  as  was  discussed 
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previously.  Again,  when  this  is  done,  care  should  be  taken 
to  assure  that  any  such  analysis  does  not  distort  reality. 

A  claeaic  example  of  such  a  situation  can  be  found  in  the 
1975  protest  of  Bell  Aerospace  Company,  where  certain 
cost/quality  ratios  were  developed  which  in  effect  wade  the 
cumulative  dollar  value  of  a  proposal  dependent  upon  the 
number  of  individual  evaluation  factors  used  in  the 
normalization  process.189  In  other  words,  the  higher  the 
number  of  evaluation  factors  used,  the  higher  the  computed 
cumulative  dollar  value. 

The  agency  regulations  examined  were  essentially 
silent  on  normalization  procedures.  In  any  case,  whether 
normalization  is  a  proper  method  of  analysis  depends  upon 
the  facts  of  the  particular  acquisition.  It  is  not 
appropriate  in  all  cases,  yet  in  some  it  should  be  done. 
Therefore,  in  creating  a  normal! ration  technique,  agencies 
should  be  intimately  familiar  with  Comptroller  General 
decisions  in  this  area. 
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CHAPTER  III 


DISCLOSURE  OF  THE  SYSTEM 


A .  Background 


Once  the  contracting  agency’s  needs  are  translated  Into 
evaluation  factors.  It  becomes  necessary  to  convey  them  In  a 
fair  and  comprehensible  manner  to  industry.  The  Comptroller 
General  has  played  the  pivotal  role  in  the  formulation  of 
the  basic  structure  of  what  is  considered  a  fair  and 
equitable  system  of  disclosure,  slowly  refining  the  system 
over  the  past  two  decades.  That  structure  is  built  around 
the  concept  of  disclosure  of  the  evaluation  factors  and 
their  relative  weights,  or  relative  importance. 

In  the  early  1960’s,  the  DAR,  FPR  and  no  doubt  most 

other  agency  regulations  contained  no  reference  to  the  need 

for  disclosure  of  evaluation  factors  and  their  relative 

weights.  After  making  several  suggestions  via  bid  protest 

decisions,  the  Comptroller  General  criticized  the  Air  Force 

in  1965  for  failure  to  disclose  pertinent  evaluation 

criteria  and  their  relative  importance  by  saying: 

With  respect  to  the  failure  of  the  Air  Force 
to  specifically  advise  prospective  offerors 
of  all  evaluation  factors  and  to  indicate 
relative  Importance  attached  to  each,  it  is 
our  opinion  that  sound  procurement  policy 
dictates  this  should  be  done.  However,  since 
we  are  aware  of  no  such  formal  requirement  we 
are  taking  occasion  to  recommend  to  the 
Secretary  of  the  Air  Force  that  procedures  in 
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this  area  be  reviewed  with  a  view  to  issuance 
of  appropriate  regulations  on  the 
subject . 190 


In  spite  of  this  clear  indication  that  the  Air  Force  had 
violated  "sound  procurement  policy",  little  regulatory 
guidance  was  promulgated.  In  1968,  another  Air  Force 
procurement  was  severely  criticized,  and  the  Comptroller 
General  "strongly"  urged  that  corrective  measures  be 
taken.191  A  year  later,  the  Comptroller  General  laid  it  on 
the  line  in  the  often-cited  protest  of  Berkeley  Scientific 
Laboratories,  Inc.192  Once  again  the  Air  Force  provided 
the  faulty  solicitation,  which,  although  a  precise  formula 
during  evaluation  was  to  be  used,  merely  stated,  "Greater 
emphasis  shall  be  placed  on  the  fallowing  criteria  in  the 
order  listed."193  Several  disclosure  requirements  were  set 
forth  in  that  opinion,  as  can  be  seen  by  the  following 
language: 

"While  we  have  never  held,  and  do  not  now 
intend  to  do  so,  that  any  mathematical 
formula  is  required  to  be  used  in  the 
evaluation  process,  we  believe  that  when  it 
is  intended  that  numerical  ratings  will  be 
employed,  offerors  should  be  informed  of  at 
least  the  major  factors  to  be  considered 
and  the  broad  scheme  of  scoring  to  be 
employed.  Whether  or  not  numerical  ratings 
are  to  be  used,  we  believe  that  notice 
should  be  given  as  to  any  minimum  standards 
which  will  be  required  as  to  any  particular 
element  of  evaluation,  as  well  as 
reasonably  definite  information  as  to  the 
degree  of  importance  to  be  accorded  to 
particular  factors  in  relation  to  each 
other. 19* 

Since  Berkeley,  the  Comptroller  General  has  continued 
to  reaffirm  the  opinions  stated  therein  and  to  refine  them. 


Later,  he  stated  the  disclosure  requirements  of  Berkelc 


applied  with  equal  force  to  civilian  agencies.*®5 
Nevertheless,  an  offeror  has  been  found  not  to  be  prejudiced 
when  he  correctly  assumed  the  relative  importance  of  the 
evaluation  criteria,  even  though  the  RFP  didn't  meet  the 
requisite  disclosure  requirements  of  the  Comptroller  General 
and  the  DAR. 1 96 

Regarding  disclosure  requirements,  DAR  3-50 1 ( b ) ( 3 ) (M) ( i ) 
states: 

[W]hen  an  award  is  to  be  based  upon 
technical  and  other  factors,  in  addition  to 
price  or  cost,  the  solicitation  shall 
clearly  inform  offerors  of  (a)  the 
significant  evaluation  factors,  and  (b)  the 
relative  order  of  importance  the  Government 
attaches  to  price  and  all  such  other 
factors.  Numerical  weights,  which  may  be 
employed  in  the  evaluation  of  proposals, 
shall  not  be  disclosed  in  solicitations;... 

Similar  requirements  are  contained  in  DODD  4105.62, 197  AFR 
70-15, 198  DARCOM  P  715-3, 199  and  NAVMAT  Instruction 
4200.49. 200  DARCOM  P  715-3  provides  some  insight,  at  least 
with  regard  to  Army  opinion,  as  to  why  numerical  weights  are 
not  to  be  disclosed,  when  it  says  that  such  disclosure  could 
create  "an  inflexible  situation  which  might  rigidify  the 
contractor's  input  to  the  detriment  of  its  overall  quality, 
and  deny  the  SSA  the  measure  of  discretion  that  he  needs  to 
protect  the  Government's  interest.”20* 

FPR  1-3 . 802 ( c ) ( 2 )  has  provisions  comparable  to  the  DAR 
regarding  disclosure  of  evaluation  criteria  and  ^their 
relative  importance,  except  that  the  FPR  does  allow 
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disclosure  in  the  solicitation  of  numerical  weights  which 
may  be  used  in  the  evaluation  of  proposals.  However,  while 
HHS  provides  that  solicitations  "need  not"  state  such 
numerical  weights,202  DOE  solicitations  are  nevertheless 
prohibited  from  disclosure  of  actual  numerical  weights 
assigned  to  evaluation  criteria.20"*  NASA  has  disclosure 
requirements  almost  identical  to  those  of  the  DAR  cited 
above,  including  the  prohibition  against  disclosure  of 
numerical  weights.204 

B.  Methods  of  Disclosure 


The  regulations  provide  scant  guidance  as  to  specific 
methods  of  disclosure  of  evaluation  criteria.  Obviously, 
the  method  chosen  to  inform  industry  of  the  relative 
importance  of  criteria  must  be  flexible  and  vary  with  the 
needs  of  the  particular  procurement.  NASA  requires  that 
evaluation  criteria  will  be  narratively  described  in  order 
of  relative  importance.20^  DOE  attempts  to  provide 
relatively  specific  guidance  by  noting  that  solicitations 
should: 

[Djescribe  the  technical,  business  and 
management  evaluation  criteria  and  clearly 
indicate  their  relative  importance.  Zt  is  not 
sufficient  to  merely  rank  the  criteria  in  terms 
of  "primary’,  'secondary',  and  next  in 
importance."  For  example,  if  the  first  of  the 
five  criteria  represents  72%  of  the  total  score 
end  six  times  the  weight  of  the  second  most 
important  criterion,  the  predominant  value  of 
that  criterion  should  be  Indicated  in  the  RFP 
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in  some  narrative  manner.  The  criteria  should 
also  be  listed  in  descending  order  of 
importance.  ?he  major  evaluation  criteria  may 
be  divided  into  sub-criteria  which  should  be 
described  in  terms  of  their  relative  importance 
and  also  be  listed  in  order  of  relative 
importance.  The  actual  numerical  weights 
assigned  to  the  evaluation  criteria  will  not  be 
included  in  the  RFP.206 


In  any  case,  no  particular  manner  of  disclosure  is 
required,  and  any  of  the  following  possible  methods  may  be 
adequate  as  long  as  the  objective  of  a  fair  and  intelligent 
competition  is  maintained. 


1 .  Absence  of  Any  Indication 


As  long  as  all  criteria  are  approximately  of 

equal  importance,  no  indication  of  relativity  need  be  made. 

In  that  same  regard,  if  the  solicitation  contains  no 

indication  of  relative  importance,  offerors  may  assume  that 

all  criteria  are  of  equal  importance.2®^  The  Comptroller 

General  recently  had  an  opportunity  to  reaffirm  his  general 

rule  in  a  1981  HEW  case,  wherein  he  held  that  when  an  RFP 

indicates  only  technical  and  cost  factors  will  be  evaluated 

for  award,  without  an  indication  of  relative  importance, 

both  factors  are  to  be  accorded  substantially  equal  weight 

in  the  evaluation. 208  The  RFP  stated: 

EVALUATION)  CRITERIA 
All  proposals  will  be  evaluated  in 
accordance  with  the  following  evaluation 
factors  and  the  respective  point  values 
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assigned  to  each  are  indicated.  Any  award 
which  may  be  made  will  be  made  to  that 
responsible  offeror  who  can  best  perform 
the  work  in  a  manner  most  advantageous  to 
the  Government,  cost  and  all  of  the  below 
factors  considered.  ( Emphasis  added . ) 20a 

HEW  conceded  that  the  relative  importance  of  cost  and 
technical  factors  was  not  specified,  and  stated  they  would 
take  corrective  actions  to  prevent  future  reoccurrences. 
Nevertheless,  the  protest  failed  on  this  point  since  HEW 
used  the  general  rule  in  their  own  defense,  i.e.,  they 
evaluated  technical  and  cost  factors  as  equal,  whereas  the 
offeror  didn't  consider  them  as  equal  in  his  offer. 

2 .  Descending  Order  of  Importance 

The  regulations  discussed  above  speak  of 
disclosure  in  terms  of  relative  order  of  importance. 
Generally,  evaluation  factors  may  be  listed  in  descending 
order  of  importance  without  otherwise  describing  their 
significance.  However,  agencies  must  be  careful  that  no  one 
factor  is  of  predominant  value.  If  one  is,  such  importance 
should  be  disclosed  to  offerors.210 

Exactly  what  is  "predominant”  is  not  an  easy 
question  to  answer.  Some  comprehension  of  the  problem  may 
be  drawn  from  two  cases.  In  the  first,  BDM  Services 


Company ,  the  O.S.  Marine  Corps  issued  an  RFP  which  listed 
five  technical  evaluation  factors,  set  forth  in  order  of 
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their  priority,  with  the  first  being  the  most  important.211 
The  established  numerical  weights  were  72,  12,  9,  4  and  3 
respectively.  BDM  protested  on  several  grounds,  one  of 
which  stated  that  they  were  misled  by  the  alleged  inadequate 
language  in  the  RFP  which  simply  stated,  "the  first  (factor) 
being  the  most  important."212  The  protest  was  found  to  be 
untimely,  but  the  Comptroller  General  commented  on  the 
Marine  Corps'  position  that  the  disclosure  followed  the 
rules  of  DAR  3-501(D)(i),  i.e.,  that  they  were  prohibited 
from  disclosure  of  numerical  weights.  He  stated  he  has 
consistently  held  that  while  offerors  should  be  informed  of 
the  relative  weight  or  importance  attached  to  evaluation 
criteria,  disclosure  of  precise  numerical  weights  is  not 
required.  Thus,  he  does  not  object  to  the  DAR  prohibition. 
However,  he  went  on  to  says 


Nevertheless,  it  has  always  been  our  position 
that  offerors  should  be  informed  -f  the 
"broad  scheme  of  scoring  to  be  employed  and 
given  reasonably  definite  information  as  to 
the  degree  of  importance  to  be  accorded  to 
particular  factors  in  relation  to  each 
other."  (Emphasis  supplied.)  *  *  *  We  have 
recognised  that  an  appropriate  method  of 
disclosing  the  relative  weights  of  the 
evaluation  criteria  is  to  list  the  evaluation 
factors  in  descending  order  of  importance  to 
priority.  *  *  *  However,  under  some 
circumstances  listing  the  evaluation  factors 
in  relative  order  of  importance  will  not 
suffice  to  even  inform  the  offerors  of  the 
broad  basis  on  which  their  proposals  are  to 
evaluated.  *  *  * 

Here  the  first  of  the  five  evaluation 
factors  listed  in  relative  order  of 
importance  constituted  72  percent  of  the 
total  technical  evaluation  score  and  was  6 
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times  the  weight  of  the  second  factor  and  24 
times  the  weight  of  the  fifth  factor.  He 
believe  that  in  consonance  with  ASPR 
3-5 0  1(D) (  i  ) ,  the  predominant  value  accorded 
the  first  factor  should  have  been  disclosed 
to  the  offerors.  Moreover,  we  believe  the 
general  relationship  of  the  remaining  factors 
to  each  other  could  have  been  described  in 
narrative  without  violating  the 
prohibition  against  disclosure  of  precise 
numerical  weights  in  ASPR  3-S01(D)(i).  As  a 
matter  of  sound  procurement  policy,  the 
fullest  possible  disclosure  of  all  of  the 
evaluation  factors  and  their  relative 
importance  is  to  be  preferred  to  reliance  on 
the  reasonableness  of  the  offerors'  judgment 
as  to  the  relative  significance  of  the 
various  evaluation  factors.21^ 


In  a  later  case,  Bayshore  Systems  Corporation,  a 
Navy  RFP  described  technical  considerations  as  the  most 
important  evaluation  factor,  and  price  as  second  most 
important.214  The  nondisclosed  numerical  evaluation 
formula  weighted  technical  factors  three  times  as  heavily  as 
price.  However,  this  inconsistency  was  found  not  to  be 
objectionable.  The  Comptroller  General  stated  that  although 
the  RFP  could  have  been  more  definite,  he  could  not  say  the 
three  to  one  relationship  was  so  out  of  line  with  the  RFP 
statement  that  use  of  the  formula  was  objectionable.  The 
RFP  clearly  pointed  out  that  technical  considerations  were 
most  important. 

The  adventuresome  individual  might  generalise  from 
the  preceding  two  cases  that  ratios  of  six  to  one  or  higher 
must  normally  be  disclosed,  whereas  ratios  of  three  to  one 
or  less  do  not.  Anything  in  between  is  open  to  conjecture. 
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However,  it  would  appear  that  this  generalisation  is  clearly 
dependent  upon  any  accompanying  language  in  the  RFP  relating 
to  relative  importance,  and  therefore  perhaps  any  such 
generalization  is  of  little  value  except  in  the  case  where 
no  narrative  is  supplied  whatsoever. 

3 .  Point  Scores  or  Percentages 

As  we  have  seen,  the  numerical  weights  which 
will  be  utilized  in  the  evaluation  can  be  used  to  indicate 
relative  importance,  except  for  NASA  and  DOD  solicitations. 
Also,  simple  percentages  may  indicate  such  importance.215 
In  one  interesting  case  the  contracting  agency  listed  the 
evaluation  factors  in  descending  order  of  importance.216 
Each  factor  was  ascribed  a  percentage  weight  in  the  RFP, 
with  a  notation  that  the  maximum  weight  would  not  exceed  a 
certain  percentage.  In  its  evaluation,  the  evaluation  panel 
varied  the  percentage  of  certain  factors  but  all  factors 
remained  in  the  same  relative  order  of  importance.  The 
Comptroller  General  denied  a  protest  against  such 
alteration,  stating  that  it  did  not  radically  depart  from 
the  RFP  evaluation  scheme. 

4 •  Adjective  or  narrative 


Relative  importance  may  also  be  indicated  by 
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assigning  adjectives,  or  by  providing  a  narrative 
description  of  each  factor's  importance.217  This  method 
can  be  and  often  is  used  in  conjunction  with  listing  on  any 
other  type  of  disclosure,  as  we  have  already  seen. 

C .  The  Relative  Importance  of  Cost/Price 

Offerors  are  entitled  to  know  the  relative  importance  of 
cost/price.  The  Comptroller  General's  often  quoted  rule  is 
that  "each  offeror  has  a  right  to  know  whether  the 
procurement  is  intended  to  achieve  a  minimum  standard  at  the 
lowest  cost  or  whether  cost  is  secondary  to  quality.*218 
This  rule  is  likewise  applicable  to  cost  reimbursement 
contracts . 2 1 9 

The  agency  regulations  examined  for  this  review  all 
recognised  the  requirement  for  and  value  of  clearly  defined 
and  disclosed  cost/technical  trade-offs.220  In  particular, 
DODD  4105.62  requires  that  all  new  major  systems  include 
design  to  cost  goals  consistent  with  the  total  cost  approach 
(i.e.,  trade-offs  among  development,  production,  operational 
and  support  costs),  and  that  such  goals  be  clearly  defined 
in  the  solicitation.221  As  such,  trade-offs  are  encouraged 
across  the  spectrum  of  design,  development,  production, 
operations,  and  support,  in  major  systems  acquisitions . 222 
While  most  of  the  regulations  simply  specify  that  such 
trade-offs  will  be  disclosed,  AF8C  Sapp  1  highlights  the 
fact  that  a  "boilerplate"  approach  to  disclosure  should  be 
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guarded  against,  and  provides  a  suggested  trade-off 
provision  for  use  as  appropriate: 


The  system  performance  requirements  contained  in 
this  Request  for  Proposal  are  structured,  to  the 
extent  practicable,  so  that  each  offeror  will  be 
free  to  propose  his  own  technical  approach,  main 
design  feature,  subsystems,  and  alternatives  to 
schedule,  cost,  and  capability  goals*  The  basic 
performance  requirements  set  forth  therein  are 
stated  as  either  firm  requirements,  as  goals,  or 
as  acceptable  parameters.  The  goals  and 
parameter  factors  may  be  traded  off  to  structure 
what  you  consider  to  be  the  optimum  balance.  To 
achieve  this  optimum  balance,  tradeoffs  of  these 
factors  may  be  made  on  a  single-element  basis  or 
in  combinations  within  the  established  parameters 
or  bands. 

The  stated  requirements,  both  firm  and  tradeable 
factors,  are  perceived  to  be  achievable  and  the 
appropriate  mixture  for  achieving  a  balanced 
application  of  all  factors  which  will  best 
service  the  Government's  needs.  If  it  is  your 
judgment  that  we  have  not  arrived  at  the  best 
mixture,  proposed  alternatives  will  be  considered 
and,  if  accepted,  the  solicitation  will  be 
revised  accordingly.223 


Obviously,  the  manner  and  intensity  of  dislcosing  the 
relative  importance  of  cost  varies  with  the  type  of 
procurement.  Particular  care  must  be  taken  in  disclosing 
the  relative  importance  of  cost/price,  as  failure  to  be 
specific  can  cause  problems. 

In  a  recent  Army  case  dealing  with  a  fixed-price 
contract,  the  RFP  listed  technical  evaluation  factors  in 
descending  order  of  importance,  and  then  merely  stated  that 
price  realism  would  be  considered.224  Poliowing 


evaluation,  the  Army  proposed  award  to  the  lowest  priced 
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technically  acceptable  offeror,  i.e.,  they  didn't  consider 

technical  superiority  advantageous  once  a  minimum  level  of 

acceptability  was  reached.  The  protestor  submitted  a 

technically  outstanding  proposal  on  the  assumption  technical 

superiority  would  be  considered  at  least  as  much  as  any 

price  advantaged  proposal.  The  Army  attempted  to  use  the 

"no  indication-equal  weight"  assumption,  to  no  avail.  The 

Comptroller  General  noted  the  Army  in  fact  did  not  intend  to 

accord  equal  weight  to  price  and  technical  factors,  but 

instead  wanted  to  award  on  the  basis  of  the  lowest  priced 

technically  acceptable  offer.  Further,  he  said  he  has 

frequently  pointed  out  that  a  reference  to  "price  and  other 

factors"  without  more  does  not  sufficiently  inform  offerors 

of  the  relative  importance  of  price,  and  stated: 

This  language  merely  establishes  that  when  making 
an  award  in  a  negotiated  procurement,  a  Source 
Selection  official  cannot  totally  disregard 
price,  50  Comp.  Gen.  110  (1970),  and  that  price 
alone  is  not  determinative  since  the  reference  to 
other  factors  includes  consideration  of  the 
technical  acceptability  of  proposals.225 

Contracting  agencies  seem  to  fare  better  with  faulty 
trade-off  disclosures  in  cost  reimbursement  contracts.  The 
Comptroller  General  distinguished  the  above  Army  case  from 
another  case  involving  a  cost  reimbursement  contract.226 
In  that  case,  even  though  the  solicitation  didn't  explicitly 
set  forth  the  relative  Importance  of  cost  to  technical 
factors,  it  was  found  to  fairly  notify  offerors  of  the 


cost-technical  trade-off,  and  that  cost  was  important 


The 
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RFP  stated  proposed  costs  would  be  considered  separately,  it 
indicated  technical  advantages  or  disadvantages  could  offset 
cost  differentials,  and  it  said  award  would  be  made  to  that 
offeror  whose  proposal  was  most  advantageous  to  the 
Government,  "price  and  other  factors  considered."227  While 
finding  the  RFP  could  have  been  more  explicit,  the 
Comptroller  General  nevertheless  stated  he  had  no  grounds  to 
conclude  that  no  reasonable  basis  for  award  existed  where 
there  was  information  indicating  both  cost  and  technical 
factors  would  be  important,  and,  that  therefore  they  were  to 
be  assumed  of  equal  importance. 

In  another  case,  the  mere  statement  that  "cost  and 
other  factors"  would  be  considered  in  award  was  declared 
insuf f icient. 22fe  However,  a  cost  reimbursement  contract 
was  involved.  In  denying  the  protest,  the  Comptroller 
General  indicated  the  FPR  provides  that  cost  estimates  are 
important  in  determining  an  offeror's  understanding  of  the 
project.  Since  the  protestor's  cost  estimate  disclosed  a 
lack  of  comprehension  of  the  basic  problem,  he  could  not 
reasonably  blame  the  RFP  for  lack  of  guidance  in  his  own 
shortcomings . 

Finally,  in  a  1980  protest,  the  Comptroller  General 
found  an  RFP  to  be  inarticulately  drafted,  but  adequate 
nevertheless.22^  When  read  in  context  of  the  entire  RFP, 
it  was  apparent  cost  was  secondary  to  technical  quality.  In 
an  example  of  poor  draftsmanship,  the  RFP  read: 
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MOTE:  Cost  is  a  factor.  However,  it  is  an 

unweighted  factor.  The  closer  the  technical 
scores  become  as  part  of  the  evaluation 
process,  the  more  significant  costs  will 
become.  230 


Later,  in  an  attempt  to  "clarify"  the  importance  of  cost, 
the  RFP  was  amended  as  follows: 


The  degree  of  the  importance  of  cost  as  an 
evaluation  factor  will  increase  with  the 
degree  of  the  quality  of  the  proposals  in 
relation  to  the  other  factors  on  which 
selection  is  to  be  based.231 


In  any  case,  the  basic  rule  in  this  area  is  clear: 
agencies  must  clearly  and  specifically  disclose  the  relative 
importance  of  cost/price  in  any  procurement.  This  is  a  rule 
founded  in  logic,  since  it  is  obviously  to  the  Government's 
advantage  to  disclose  those  areas  where  acceptable  savings 
may  be  made.  The  most  problems  appear  to  occur  in  the 
drafting  of  those  trade-offs  in  the  RFP.  Thus,  agencies 
must  strive  to  be  as  precise  and  as  specific  as  possible  in 
their  disclosures. 


D.  Disclosure  of  Subcriteria 


The  agency  regulations  surveyed  offer  relatively  little 
guidance  regarding  disclosure  of  subcriteria.  Those 
regulations  that  do  comment  on  subcriteria  disclosure  simply 
reiterate  the  basic  rules  discussed  below.  However,  NASA 
curiously  provides  that  subcriteria  will  not  be  included  in 
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the  solicitation. 232  As  will  be  seen,  the  extent  of  this 
NASA  provision  is  limited  by  Comptroller  General  rulings. 
Generally,  subcriteria  may  be  listed  in  solicitation,  but 
there  is  no  requirement  to  do  so  if  the  subcriteria  are 
logically  and  reasonably  related  to  the  stated  evaluation 
factors.233  An  apparent  exception  to  that  general  rule  was 
made  in  the  1976  protest  of  Dikewood  Services  Company,  where 
the  Comptroller  General  stated: 


Each  subcriteria  need  not  be  disclosed  so  long 
as  offerors  are  advised  of  the  basic  criteria, 
and  any  subcriteria  used  by  the  agency  in  the 
actual  evaluation  are  merely  definitive  of  the 
basic  criteria.  However,  where  a  relatively 
sketchy  evaluation  plan  is  stated  in  the  RFP, 
and  the  agency  possesses  an  extremely  detailed 
evaluation  scheme  with  numerous,  unannounced, 
definitive  subcriteria,  the  withholding  of 
those  known  subcriteria  does  not  promote  the 
basic  procurement  objective  of  providing 
offerors  with  sufficient  information  to  prepare 
an  intelligent  response  to  the  Government’s 
requirements . 234 


Thus,  although  definitive  subfactors  need  not  normally  be 
disclosed,  it  appears  that  when  an  agency  uses  an  extremely 
detailed  evaluation  scheme  with  numerous,  undisclosed 
subfactors,  and  where  the  RFP  is  sketchy,  those  subfactors 
should  be  revealed  to  assure  intelligent  offers. 

In  those  cases  where  subfactors  should  be  disclosed,  the 
question  arises  as  to  whether  or  not  their  relative  weights 
need  be  disclosed  also.  The  Comptroller  General  addressed 
this  question  in  the  RCA  Service  Company  protest  discussed 
at  length  in  the  beginning  of  Chapter  IX.  In  that  case,  the 
RFP  listed  six  subcriteria  under  the  first  main  criterion. 


but  it  contained  no  relative  weights  for  then 
sixth  subfactor  became  critical  in  the  award. 
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As  noted, the 
Finding  an 

impropriety  therein,  the  Comptroller  General  ruled  that  when 
the  solicitation  contains  no  information  on  the  relative 
weights  of  subfactors,  offerors  are  entitled  to  consider 
them  to  be  of  equal  importance.  Thus,  offerors  must  be  told 
of  unequally  weighted  subcriteria. 

The  RCA  ruling  was  modified  somewhat  in  a  1979  protest 
by  AEL  Service  Corporation,  et  al.235  That  protest 
involved  the  Army's  failure  to  disclose  all  of  the 
subfactors  used  in  their  evaluation,  and  their  failure  to 
disclose  the  relative  weights  of  all  disclosed  subfactors. 

As  to  the  failure  to  disclose  all  of  the  subcriteria,  the 
Comptroller  General  applied  the  "logically  and  reasonably 
related"  rule  to  find  that  their  disclosure  was  not 
required.  As  to  the  relative  weights  of  the  disclosed 
subfactors,  he  had  this  to  say: 

In  regard  to  AEL's  contention  concerning 
deception  as  to  the  relative  weights  of  the 
evaluation  criteria,  it  has  been  the 
consistent  position  of  our  Office  that 
offerors  should  be  placed  in  a  position  to 
make  accurate  and  realistic  proposals  by 
informing  them  of  relative  importance  in  the 
solicitation.  *  *  * 

Conceding  the  efficacy  of  this  principle,  we 
believe  that  its  effect  should  be  limited  to 
the  principal  evaluation  factors.  *  *  *  That 
is  to  say,  not  all  subcriteria  of  the 
principal  factors  need  to  be  treated  in  the 
same  manner  as  the  principal  criteria.  *  *  * 
Being  definitive  sibcriteria,  we  find  no  harm 
in  the  failure  to  disclose  the  relative 
weights  of  these  types  of  subc  eerie,  as 
they  are  all  elements  vhich  basically 
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comprise  the  main  criteria,  but  in  a 
narrative  fashion.  It  is  our  opinion  that  an 
offeror  could  not  realistically  assume  that 
subcriteria  of  such  a  definitive  nature, 
unless  stated  otherwise,  would  be  of  equal 
importance  in  relation  to  each  other.  *  *  * 

We  do  however,  want  to  clarify  and 
distinguish  this  position  from  instances 
involving  subcriteria  which  are  essential 
characteristics  or  measurements  of 
performance  of  the  end  item  being  procured. 

We  believe  the  relative  importance  of  these 
subcriteria  would  have  to  be  disclosed  in 
order  to  allow  offerors  to  properly  formulate 
their  proposals. 

Our  holding  in  51  Comp.  Gen.  272,  281,  is 
modified  to  the  extent  inconsistent  with  the 
foregoing. 236 


Thus,  the  curious  result  of  this  decision  appears  to  be  that 
if  the  disclosed  subcriteria  contain  no  relative  weights, 
they  may  be  assumed  equal  only  when  they  are  "essential", 
since  if  they  are  "definitive",  they  could  be  of  unequal 
weight . 


X.  Disclosure  of  Evaluation  Methods 


with  the  exception  of  NASA  and  the  Army,  none  of  the 
regulations  reviewed  mentioned  disclosure  of  the  actual 
method  of  evaluation  to  be  employed.  NASA  requires  the 
solicitation  to  contain  a  clear  explanation  of  the  method  of 
evaluation  to  be  used,  "so  that  prospective  offerors  may 
understand  the  SEB's  use  and  treatment  of  the  four 
categories  of  factors,  and  so  that  they  will  know  that  the 
SSO ,  not  the  8EB,  will  make  the  judgments  required,  all 
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factors  considered,  in  selectin9  the  winning  offeror.*237 

The  Army  similarly  requires  a  disclosure  of  the  general 

method  of  scoring  to  be  employed,  the  rationale  for  such 

requirement  being  stated  as  follows: 

[0] f ferors  sometimes  misunderstand  how  their 
proposals  are  evaluated,  particularly  when  both 
numerical  and  qualitative  scoring  techniques 
are  mixed  in  an  evaluation.  Such  misunder¬ 
standings  can  be  avoided  with  a  brief 
description  of  the  scoring  plan  to  be  employed. 
Although  there  is  no  preference  relative  to  the 
various  methods  of  factor  scoring,  a  broad 
scheme  of  scoring  is  appropriate  for  disclosure 
to  the  offerors.238 


In  any  case,  there  is  no  requirement  to  disclose  the 
method  of  evaluation  to  be  utilized.  In  a  1981  protest,  the 
protestor  objected  to  the  evaluation  method  used  by  the 
Navy,  arguing  in  part  that  since  they  were  not  told  of  the 
manner  in  which  the  evaluation  would  be  conducted,  the  Navy 
was  not  in  compliance  with  the  applicable  procurement 
regulations  and  past  Comptroller  General  decisions  which 
required  disclosure  of  evaluation  factors.239  Disagreeing, 
the  Comptroller  General  stated: 


The  decision  of  our  Office  cited  by  Ridgeway, 
48  Comp.  Gen.  464  (1969),  and  DAR  $2-503.1, 
require  that  a  procuring  agency  disclose  the 
evaluation  factors  to  be  used  and  their 
relative  weights,  not  the  evaluation  method. 

*  *  *  We  find  the  RFTP  adequately  advised 
offerors  of  the  manner  in  which  proposals 
were  to  be  prepared  and  the  manner  in  which 
the  proposals  were  actually  evaluated  would 
have  had  no  impact  on  a  properly  prepared 
proposal.240 
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CHAPTER  IV 


0SE  OF  THE  SYSTEM 

A.  Consistency 

Chapter  II  began  with  the  proposition  that  the  one 
thread  which  binds  the  entire  source  selection  process 
together  was  that  of  agency  discretion.  That  discretion  was 
present  in  the  creation  of  a  system,  in  its  disclosure,  and 
finally,  in  the  actual  use  of  the  system  in  making  the 
ultimate  selection.  In  exercising  that  discretion  in 
selection,  agency  procurement  officials  are  subject  only  to 
the  tests  of  "rationality  and  consistency  with  established 
evaluation  factors."241  The  former  test  will  be  dealt  with 
in  the  next  section  of  this  chapter.  The  latter  test 
maintains  that  proposals  should  be  evaluated  on  the  basis  of 
the  criteria  set  forth  in  the  solicitation.  If  there  are 
material  changes  to  the  disclosed  criteria,  then  the 
solicitation  should  be  amended,  and  all  offerors  should  be 
given  the  opportunity  to  revise  their  proposals • 242  The 
procurement  regulations  recognize  these  requirements.  DAR 
3-805.4  provides: 

(ir]hen,  either  before  or  after  receipt  of 
proposals,  changes  occur  in  the  Government’s 
requirements  or  a  decision  is  made  to  relax. 
Increase  or  otherwise  modify,  the  scope  of  the 
work  or  statement  or  requirements,  such  change 
or  modification  shall  be  made  in  writing  as  an 
amendment  to  the  solicitation. 


The  FPR243  and  NASA  PR244  contain  similar  requirements,  as 
do  the  particular  agency  regulations  surveyed. 24®  Clearly, 
the  reason  for  these  requirements  lies  in  the  statutory  and 
regulatory  objectives  of  promoting  maximum  practicable 
competition.  These  objectives  can  only  be  met  if  all 
offerors  are  in  fact  competing  on  an  equal  basis,  such  basis 
being  composed  of  common  evaluation  criteria. 

The  Comptroller  General  has  had  frequent  opportunity  to 
examine  protests  invoking  the  alleged  use  of  undisclosed  or 
changed  evaluation  criteria.  Simply  stated,  agencies  create 
their  own  problems  by  not  applying  the  evaluation  factors  as 
disclosed.  For  example,  an  agency's  evaluation  was  found 
improper  where  they  ignored  six  out  of  twelve  disclosed 
subfactors,  the  effect  of  which  increased  the  relative 
importance  of  price  from  an  intended  30%  to  an  actual 
50%. 246  As  discussed  previously,  it  was  improper  for  an 
agency  to  use  a  formula  in  evaluating  proposals  which  gave 
only  negligible  weight  to  cost,  when  the  solicitation 
indicated  cost  would  be  given  an  evaluation  weight  of 
20% . 247 

Criteria  modifications  and  additions  during  evaluations 
provide  fertile  ground  for  protests.  Where  prospective 
contractors  were  informed  in  a  solicitation  that  award  would 
be  based  on  a  rental  price  per  square  foot  (among  other 
factors) ,  an  evaluation  and  award  based  on  an  undisclosed 
factor  Involving  lowest  overall  life  cycle  coat  was 
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faulty.248  The  Army  had  problems  with  life  cycle  costs 
when  they  evaluated  them  over  a  life  period  of  12  months, 
notwithstanding  the  fact  that  the  solicitation  said 
evaluation  would  encompass  a  60  month  period.248  The  Army 
failed  to  convince  the  Comptroller  General  that  under  any 
life  period  the  awardee  would  have  been  low  since  the  cost 
evaluators’  scoring  was  subjective,  and  it  had  disparities. 
Therefore,  it  couldn't  be  accurately  determined  how  the 
evaluators  would  have  scored  cost  if  they  had  used  a  60 
month  period.  HEW  found  similar  criticism  by  requesting 
offers  for  performance  of  a  contract  over  a  17  1/2  month 
period,  yet  awarding  to  a  firm  on  a  27  months  basis.250 

The  improper  use  of  different  criteria  than  that 
disclosed  in  the  solicitation  can  clearly  be  recognized  in 
one  protest  where  the  agency  informed  the  prospective 
contractors  that  primary  emphasis  in  a  study  covering 
several  environmental  media  would  be  on  drinking  water, 
while  in  fact  the  intent  of  the  agency  was  to  place  equal 
emphasis  on  each  of  several  disclosed  media.251  The 
Comptroller  General  found  material  prejudice  in  the  failure 
to  clarify  such  known  requirements  by  issuing  a  written 
amendment  after  receipt  of  initial  proposals. 

Other  examples  of  inappropriate  uses  of  undisclosed 
criteria  include  consideration  of  the  actual  operation  of  a 
system  while  the  RFP  indicated  only  the  evaluation  of  a 
capacity  for  such  system,252  and  award  to  an  offeror  whose 
offered  price  would  become  low  price  only  upon  the  agency's 
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exercise  of  an  option,  when  the  solicitation  didn't  provide 
for  the  evaluation  of  options.233  The  latter  award  was 
declared  to  be  arbitrary  and  capricious,  and  without  a 
reasonable  basis.  Changes  in  delivery  schedules  may  also 
require  amended  solicitations.23*  In  perhaps  the  highest 
valued  protest  ever  sustained  by  the  Comptroller  General 
(approximately  $1  Billion),  the  Air  Force's  acceptance  of  an 
alternate  delivery  schedule  that  deviated  from  the 
solicitation  was  held  an  impermissible  material  change  which 
required  notice  to  all  offerors  and  opportunity  for  revision 
of  proposals.233  The  Comptroller  General  found  that  the 
awardee's  proposed  accelerated  delivery  schedule  represented 
a  relaxation  of  the  stated  RFP  requirements,  noting  that  the 
delivery  of  some  of  the  components  pursuant  to  the  awardee's 
schedule  could  be  made  at  a  later  date  than  required  by  the 
RFP.  This  defect  was  considered  serious,  and  the 
Comptroller  General  recommended  the  award  *>e  set  aside. 
However,  that  recommendation  was  rejected  by  a  federal 
district  court  judge,  who  found  that  the  decision  to  award 
had  a  rational  basis,  and  that  national  security  interests 
precluded  disturbing  the  award.23® 

Procuring  agencies  seem  to  overcome  allegations  that 
they  have  used  undisclosed  criteria  improperly  if  those 
undisclosed  criteria  bear  some  reasonable  relationship  to 
appropriate  disclosed  criteria.  Perhaps  this  is  simply  a 
somewhat  awkward  analogy  to  the  rule  regarding  the 
disclosure  of  subcriteria  discussed  previously.  For 
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example.  In  a  1981  protest,  the  Army  ased  an  evaluation 
factor  not  in  the  RFP  which  assigned  more  weight  to  a 
proposal  containing  experts  in  both  Marine  Corps  and  Navy 
aspects  of  helicopter  aviation,  than  to  one  with  a  single 
expert  in  both  aspects.257  The  plan  was  held  to  be 
appropriate,  since  the  importance  and  experience  of  such 
experts  were  "reasonably  discernible"  from  the  RFP.  As 
such,  "offerors  were  on  notice  and  should  have  anticipated 
this  subcriterion."258  Likewise,  DOE  successfully  avoided 
a  protest  against  their  use  of  a  mock  task  in  its  evaluation 
process.259  The  RFP  contained  no  criteria  for  evaluating 
the  mock  task,  nor  did  it  have  any  indicia  of  its 
significance.  The  task  was  not  point  scored  nor  adjectively 
rated,  but  did  impact  upon  the  SEB's  "overall  perception  of 
each  offer  in  light  of  the  evaluation  criteria."260  In 
finding  the  protest  without  merit,  the  Comptroller  General 
stated  that  the  mock  task  constituted  a  requisite 
demonstration  of  timely  ability  to  respond  to  certain 
assignments,  and  a  demonstration  of  each  project  manager's 
ability  to  command  required  resources,  both  of  which  were 
disclosed  in  the  RFP. 

Two  other  examples  of  "reasonably  discernible"  criteria 
are  where  offerors  were  adequately  informed  that  700  points 
would  be  assigned  to  price  and  500  points  to  technical 
operations  when  the  RFP  stated  each  would  be  "almost  equal 
in  weights",261  and  where  the  use  of  education,  experience 
and  salaries  when  evaluating  an  "understanding  of  personnel 
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requirements”  criterion  was  declared  sufficiently  related  to 
the  solicitation  criteria.262 

One  unusual  case  concerned  a  situation  where  a  source 
selection  authority  was  unable  to  make  a  reasonable  decision 
when  two  proposals  were  evaluated  as  totally  equal.  The 
Comptroller  General  examined  both  offers  carefully  and 
agreed  that  the  SSA  had  to  find  an  appropriate 
discriminator.  In  this  case,  they  consisted  of  two  factors: 
the  disruptive  effects  and  cost  consequences  of  not  awarding 
to  an  incumbent  contractor,  and  the  status  of  one  offeror  as 
a  labor  surplus  area  concern,  neither  of  which  were 
encompassed  in  the  RFP.  The  Comptroller  General  explained 
his  reasoning: 


When,  however,  competing  proposals  are 
measured  against  the  evaluation  factors 
established  for  the  procurement  and  the 
source  selection  official,  in  the  good 
faith  exercise  of  the  discretion  vested  in 
h'm,  is  unable  to  discern  an  appropriate 
choice  on  the  basis  of  that  evaluation,  we 
think  that  official  properly  may  take  into 
account  other  factors  which  are  rationally 
related  to  a  selection  decision  for  the 
particular  procurement  involved.264 


On  occasion,  proposals  are  rated  equal  in  technical 
merit  by  evaluation  personnel.  When  this  occurs,  cost/price 
can  become  the  determinative  factor  notwithstanding  the 
fact  that  technical  factors  are  stated  in  the  solicitation 
to  be  of  primary  importance.  This  situation  often  gives 
rise  to  protests  alleging  that  cost/price  has  thus  been 
given  more  weight  than  disclosed  in  the  solicitation.  In 
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addressing  such  allegations,  the  Comptroller  General  has 
held  that  the  utilization  of  cost/price  as  the  determinative 
factor  in  such  instances  does  not  mean  the  agency  has 
altered  the  relative  importance  of  their  disclosed 
criteria.265 

In  one  protest,  an  RFP  indicated  technical 

considerations  were  worth  85  points  while  price  was  valued 

at  35  points.266  Also,  it  stated  award  would  be  made  to 

the  technically  acceptable  proposal  offering  the  most 

advantageous  technical/cost  relationship,  i.e.,  not  to  the 

highest  point  score  attained  by  an  offeror.  The  agency 

determined  two  offers  were  essentially  equal  technically, 

and  awarded  to  the  offeror  with  the  lowest  proposed  price. 

Rejecting  a  contention  that  there  was  a  change  in  the  stated 

evaluation  criteria  as  a  result  of  that  action,  the 

Comptroller  General  related: 

In  any  case  where  cost  is  designated  as  a 
relatively  unimportant  evaluation  factor, 
it  may  nevertheless  become  the 
determinative  factor  when  application  of 
the  other,  more  important  factors  do  not, 
in  the  good  faith  judgments  of  source 
selection  officials,  clearly  delineate  a 
proposal  which  would  be  most  advantageous 
to  the  Government  to  accept.267 

However,  such  determinations  of  equality  of  proposals  must 
be  supported  in  the  evaluation  record.  Thus,  HKff  was 
criticised  in  1979  for  failing  to  provide  a  factual 
explanation  why  a  review  committee  concluded  a  difference  of 
11.75  points  out  of  100  rendered  two  proposals  essentially 
equal  in  technical  merit.268  Comptroller  General  found 
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the  mere  inclusion  of  conclusionary  statements  to  be 
unacceptable . 

Finally,  offerors  have  found  that  the  general  rule  in 
this  area  may  work  against  them.  The  Comptroller  General 
dismissed  a  contention  that  an  awardee’s  proposal  should 
have  been  rejected  because  it  didn't  meet  certain  alleged 
requirements  when  he  found  that  the  RFP  in  fact  imposed  no 
such  requirements  and  that  HEW  applied  proper  criteria  in 


their  evaluation. 
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B .  Rationality 

Assuming  established  and  disclosed  factors  have  been 
consistently  applied,  agency  source  selections  must  meet  the 
test  of  rationality.  Once  again,  the  Source  Selection 
Official  or  Authority  is  vested  with  wide  discretion  and  is 
not  bound  by  the  rankings  or  recommendations  (if  any) 
provided  by  evaluation  boards,  as  long  as  there  is  a 
rational  basis  for  the  decision.270  Point  spreads,  as  we 
have  seen,  are  guides  for  the  source  selection  officials, 
and  the  fact  that  an  agency  may  have  used  a  numerical 
scoring  system  and  assigned  somewhat  different  scores  to 
competing  proposals  doesn't  mean  a  higher  rated  proposal 
must  be  viewed  as  offering  any  significant  technical 
advantage . 27 *  Therefore,  a  point  spread  of  99.5,  96.0  and 
93.8  out  of  a  possible  100  points  was  properly  deemed  to  be 
"substantially  equal"  by  source  selection  personnel.272 


Indeed,  even  if  technical  evaluators  unanimously  recommend 
award,  the  source  selection  official  may  nevertheless 
exercise  his  discretion  and  choose  a  different  offeror  as 
long  as  he  has  a  reasonable  basis  and  is  consistent  with 
evaluation  criteria.273 

In  any  case,  the  procurement  record  must  clearly  show 
that  there  was  in  fact  a  rational  basis  for  the  award 
decision.  Thus,  one  record  was  insufficient  where  it 
contained  no  indication  of  the  reasoning  process  underlying 
the  bare  conclusions  of  the  third  of  three  evaluators,  whose 
views  prompted  the  final  selection,  and  where  those  views 
conflicted  with  the  technical  evaluation  committee's 
views.274  The  extent  of  such  justification  necessary  to 
support  an  award  obviously  varies  with  the  circumstances  of 
the  particular  procurement.  For  example,  when  there  is  only 
a  slight  difference  in  technical  scores,  the  fact  that  a 
lower  rated  offeror  submitted  a  substantially  lower  cost 
proposal  may  in  itself  be  adequate  to  support  an  award.275 
Sometimes  the  technical  evaluation  reports  themselves  may 
provide  sufficient  justification,275  while  other 
justifications  require  detailed  analysis.277  In  that 
regard,  a  protest  was  sustained  where  two  offerors  were 
rated  in  technical  excellence  at  97  and  87  out  of  a  possible 
100  points,  and  at  17  and  20,  respectively,  out  of  a 
possible  20  points  for  cost.278  Award  was  made  to  the  less 
expensive  offeror,  even  though  he  was  rated  lower 
technically.  The  protestor  argued  that  such  -m  decision  made 
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price  the  primary  consideration,  rather  than  technical 
excellence,  as  was  stated  in  the  solicitation.  The 
Contracting  Officer  replied  that  the  technical  scores 
indicated  only  a  "slight  difference"  between  offerors. 
However,  the  record  only  contained  the  composite  score 
sheets  for  the  SEB  members,  with  no  individual  ratings  or 
narrative  summaries.  The  Comptroller  General  found  such  a 
record  to  be  insufficient  to  establish  the  rationality  of 
the  Contracting  Officer's  decision  that  the  10  point 
differential  was  insubstantial.  Finally,  a  selection  may  be 
unreasonable  on  its  face,  as  where  award  is  made  to  an 
offeror  whose  technical  proposal  was  5%  higher,  but  whose 
price  was  4  1/2  times  higher.279 

As  for  the  agency  regulations  examined  in  this  review, 
they  offer  little  instruction  regarding  the  actual  selection 
decision.  AFR  70-15  simply  'requires  that  the  selection 
decision  be  incorporated  into  a  "Source  Selection  Decision 
Document"  which  must  contain  a  detailed  rationale  for  the 
selection.280  The  Army  points  out  that  the  SSA  is  not 
bound  by  the  "findings,  scoring,  and  recommendations  of 
lower  level  review  bodies  and  officials.”281  The  limits  on 
the  SSA's  discretion  are  said  to  be  that  the  award  decision 
must  have  "a  rational  basis  in  terms  of  the  evaluation 
factors  in  the  RFP,  and  all  the  legal  and  procedural 
requirements  of  the  evaluation  process  must  be  met..."282 
Also,  the  final  decision  should  not  be  based  on  "numerical, 
adjectival  or  other  scores,  rather,  it  should  be  based  upon 
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the  relative  strengths  and  weaknesses  of  the  competing 

proposals."283  NASA  provides  thatt 

In  the  final  analysis,  NASA  judgment  on  the 
totality  of  the  evaluation  will  be  that  of 
the  Source  Selection  Official.  This  includes 
assessment  of  the  procedures  followed  by  the 
Board,  the  validity  of  its  substantive 
evaluations,  the  relative  significance  of  the 
several  areas  of  the  evaluation,  and  the 
weightings  previously  assigned  by  the  SEB,  in 
the  light  of  all  the  information  produced  by 
the  source  evaluation  and  selection  process. 
The  Source  Selection  Official  will  select  the 
contractor  (or  contractors)  which  he 
considers  can  perform  the  contract  in  a 
manner  most  advantageous  to  the 
Government.284 


Obviously,  one  can  only  describe  agency  discretion  in  so 
many  ways. 

While  allegations  of  irrationality  may  be  found  in  areas 
other  than  cost,  such  as  the  RCA  protest  cited  in  Chapter 
II,  the  inherent  problems  of  cost/technical  trade-offs 
provide  the  most  fertile  grounds  for  protests  against  agency 
selections.  The  extent  to  which  such  trade-offs  may  be  made 
is  governed  by  the  evaluation  scheme  set  forth  in  the 
solicitation. 283  Where  cost  and  technical  factors  are 
afforded  substantially  equal  weight,  award  to  a  higher  cost, 
higher  technically  rated  offer  is  proper  when  it  reasonably 
provides  a  better  value  to  the  Government.288  Thus,  there 
were  no  grounds  to  protest  a  case  where  cost  and  technical 
factors  were  rated  equally,  and  the  awardee’s  cost  was  2.4% 
higher  than  the  protester's,  but  at  the  same  time  44.9% 
higher  technically.287 
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As  was  discussed  previously,  coat  considerations  can 
become  determinative,  even  though  of  minimal  relative 
importance,  where  proposals  are  evaluated  as  being 
technically  equal.  Again,  such  determinations  of  equality 
must  likewise  be  supported  as  reasonable  and  rational.288 
As  such,  where  a  solicitation  places  emphasis  on  technical 
merit,  and  the  record  doesn't  justify  the  "essentially 
equal"  determination,  award  based  upon  lower  estimated  cost 
is  improper.289  In  that  regard,  the  Comptroller  General 
has  stated: 


We  believe  that  implicit  in  the  language  of 
the  RFP  that  "cost  will  be  considered 
secondary  to  technical  merit",  is  an 
invitation  to  offerors  to  propose  the  use 
of  methods,  facilities  and  resources  which 
they  believe  will  best  accomplish  the 
desired  result,  not  necessarily  at  lowest 
cost,  but  a  cost  to  the  Government  which  is 
fair  and  reasonable.290 


C .  Discretion:  A  Final  Word 


This  review  began  by  examining  the  RCA  protest  in  detail 
in  order  to  gain  an  overview  of  the  source  selection  system 
and  its  binding  thread:  agency  discretion.  Not  surprising¬ 
ly,  this  review  could  be  ended  by  citing  the  RCA  case. 

Many  aspects  of  the  source  selection  process  have  been  dealt 
with  in  this  review,  all  linked  by  a  common  denominator. 

The  decisional  discretion  vested  in  procuring  agency 


officials  is  real  and  extensive.  No  better  summarisation  of 
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the  range  of  that  discretion  can  be  found  than  that  set 

forth  by  the  Comptroller  General  in  the  protest  of  Grey 

Advertising,  Inc.,  where  he  stated: 

[We]  have  consistently  stated  that  "technical 
point  ratings  are  useful  as  guides  for 
intelligent  decision-making  in  the 
procurement  process,  but  whether  a  given 
point  spread  between  two  competing  proposals 
indicates  the  significant  superiority  of  one 
proposal  over  another  depends  upon  the  facts 
and  circumstances  of  each  procurement  and  is 
primarily  a  matter  within  the  discretion  of 
the  procuring  agency.  52  Comp.  Gen.  686,690 
(t973)i  52  id.  738,  747  (1973);  ILC  Dover, 

B- 182104,  November  29,  1974,  74-2  CPD  301; 

Tracor  Jitco,  Inc.,  54  Comp.  Gen.  896  (1975), 
75-1  CPD  253;  Management  Services 
Incorporated,  B-184606,  Pebruary  5,  1976,  55 

Comp.  Gen.  _ _ ,  76-1  CPD  74.  As  we  said  in 

Tracor  Jitco,  Inc. ,  supra ; 

"*  *  *  Uniformly,  we  have  agreed  with 
the  exercise  of  the  administrative 
discretion  involved--in  the  absence  of 
a  clear  showing  that  the  exercised 
discretion  was  not  rationally 
founded— a*  to  whether  a  given 
technical  point  spread  between 
competitive-range  offerors  showed  that 
the  higher-scored  proposal  was 
technically  superior.  On  a  finding 
that  technical  superiority  was  shown 
by  the  point  spread  and  accompanying 
technical  narrative,  we  have  upheld 
awards  to  concerns  submitting  superior 
proposals,  although  the  awards  were 
made  at  costs  higher  than  those 
proposed  in  technically  inferior 
proposals.  52  Comp.  Gen.  358  (1972); 
B- 171696,  July  20,  1971;  B-170633,  May 
3,  1971.  Similarly,  on  a  finding  that 
the  point  score  and  technical 
narrative  did  not  indicate  superiority 
in  the  high-ranked  proposal,  we  have 
upheld  awards  to  offerors  submitting 
less  costly,  albeit  lower-scored 
technical  proposals.  See  52  Comp. 

Gen.  686  (1973);  50  id.,  supra.  This 
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reflects  our  view  that  the  procuring 
agency's  evaluation  of  proposed  costs 
and  technical  approaches  are  entitled 
to  great  weight  since  the  agencies  are 
in  the  best  position  to  determine 
realism  of  costs  and  corresponding 
technical  approaches.  Matter  of 
Raytheon  Company,  54  Comp.  Gen.  189 
(1974);  50  id.  390  (1970).  Our 
practice  of  deferring  to  the  agency 
involved  in  cost/technical  trade-off 
judgments  has  been  followed  even  when 
the  agency  official  ultimately 
responsible  for  selecting  the 
successful  contractor  disagreed  with 
an  assessment  of  technical  superiority 
made  by  a  working  level  evaluation 
committee.  See  B-173137(1),  October  8, 
1971.  Our  review  of  the  subject 
award,  therefore,  is  limited  to 
deciding  whether  the  record  reasonably 
supports  a  conclusion  that  the  award 
was  rationally  founded.  See  Matter  of 
Vinnell  Corporation,  B-180557,  October 
8,  1974."  54  Comp.  Gen.  at  898-9. 

We  belie  e  it  is  clear  from  these  cases 
that  t  s  question  of  whether  a  difference 
in  point  scores  is  for  determination  on  the 
basis  of  both  what  the  difference  might 
mean  in  terms  of  performance  and  what  it 
would  cost  the  Government  to  take  advantage 
of  it.  As  we  said  in  52  Comp.  Gen.  358 
(1972),  the  "determinative  element  *  *  * 
[is]  not  the  difference  in  technical  merit 
scores  per  se ,  but  the  considered  judgment 
of  the  procuring  agency  concerning  the 
significance  of  that  difference."  52  Comp. 
Gen.  at  365.  Thus,  for  example,  in 
B-173137(1),  October  8,  1971,  where  it  was 
determined  that  two  firms  were  technically 
equal  despite  one  firm's  technical  point 
score  edge  of  15.8  points  (on  a  100  point 
scale) ,  we  viewed  the  award  to  the 
lower-scored  competing  firm  "as  evidencing 
a  determination  that  the  cost  premium 
involved  in  making  an  award  to  [the 
higher-rated]  firm  would  not  be  justified 
in  light  of  the  acceptable  level  of 
technical  competence  available  at  somewhat 
lower  cost."  See  also  50  Comp.  Gen.  246 
(1970),  in  which  we  expressed  similar  views 
in  upholding  award  to  a  firm  receiving  a 
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point  score  that  was  6  points  (out  of  100) 
lower  than  that  received  by  a  competitor. 
Also,  in  ILC  Dover,  supra ,  we  upheld  a 
contracting  officer's  determination  that  a 
point  spread  of  2.75  out  of  100  “was 
'insufficient  in  the  light  of  the 
substantially  higher  cost'  associated  with 
the  protester's  proposal." 

On  the  other  hand,  we  have  upheld  an  award 
to  a  high-rated  (14  points  out  of  100) 
offeror  with  significantly  higher  proposed 
costs  because  we  viewed  the  award  "as 
reflecting  a  determination  that  the  cost 
premium  involved  was  justified  taking  into 
account  the  significant  technical 
superiority  of  [the  winning  offeror's] 
proposal."  b-170181,  February  22,  1971. 

See  also  52  Comp.  Gen  358,  supra  (where 
technically  superior  offeror  was  rated  3 
points  higher  than  a  competing  firm); 
Riggins  &  Williamson  Machine  Company, 
Incorporated,  et  al.,  54  Comp.  Gen.  783 
(1975),  75-1  CPD  783;  Planning  Research 
Corporation ,  B-182962,  July  15,  1975,  75-2 

CPD  37;  Bellmore  Johnson  Tool  Company, 

B- 179030,  January  24,  1974,  74-1  CPD  26. 
However,  where  award  was  made  to  an  offeror 
whose  technical  proposal  was  scored  about  5 
percent  higher  than  a  competitor's 
technical  proposal  but  whose  price  was 
approximately  f our-and-one-half  times 
higher,  we  said  the  record  did  not 
indicate  that  the  technical  superiority  of 
the  one  offeror  "warranted  an  award  to  him 
at  a  substantially  high  price"  and  that 
therefore  the  record  did  not  support  the 
conclusion  that  the  award  made  was  most 
advantageous  to  the  Government.  Design 
Concepts ,  Inc . ,  B-184658,  January  23,  1976, 
76-1  CPD  39. 

Furthermore,  while  point  scores,  technical 
evaluation  narratives,  and  adjective 
ratings  may  well  be  indicative  of  whether 
one  proposal  is  technically  superior  to 
another  and  should  therefore  be  considered 
by  source  selection  officials,  see  EPSCO, 
Incorporated,  B-183816,  November  21,  1975, 
55  Comp.  Gen .  ,  75-2  CPD  338,  we  have 

recognised  that  selection  officials  are  not 
bound  by  the  recommendations  made  by 
evaluation  and  advisory  groups.  Bell 
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Aerospace  Company,  55  Comp.  Gen.  244 
(1975),  75-2  CPD  168;  Tracor  Jltco,  Inc., 
supra ?  51  Comp.  Gen.  272  (1971  )» 
8-173137(1),  supra .  This  is  so  even  though 
it  is  the  working  level  procurement 
officials  and  evaluation  panel  members  who 
may  normally  be  expected  to  have  the 
technical  expertise  relevant  to  the 
technical  evaluation  of  proposals. 

Accordingly,  we  have  upheld  source 
selection  officials'  determinations  that 
technical  proposals  were  essentially  equal 
despite  an  evaluation  point  score 
differential  or  81  out  of  1000,  see  52 
Comp.  Gen.  686,  supra ,  and  despite 
contracting  officer  recommendations  that 
award  be  made  to  the  offeror  with  the 
highest  technical  rating.  See  52  Comp. 

Gen.  738,  supra . 

As  indicated  by  the  foregoing,  source 
selection  decision-making  is  vested  in  the 
"considerable  range  of  judgment  and 
discretion"  of  the  selection  officials, 
EPSCO,  Incorporated,  supra ,  who  have  a 
"very  broad  degree  of  discretion  *  *  *  in 
determining  the  manner  and  extent  to  which 
[they]  will  sake  use  of  technical 
evaluation  results.”  Department  of  Labor 
Day  Care  Parents'  Association,  54  Comp. 

Gen.  1035,  1040  (1975),  75-1  CPD  353.  In 
exercising  that  discretion,  they  are 
subject  only  to  the  tests  of  rationality 
( see  Tracor  Jitco,  Inc.,  supra ,  where  we 
questioned  the  selection  decision  as  not 
■rationally  justified"  but  ultimately  found 
in  a  later  decision  on  the  basis  of  a 
subsequent  submission  from  the  procuring 
agency,  that  the  selection  decision  was 
supportable.  55  Comp.  Gen.  499  (1975), 

75-2  CPD  344)  and  consistency  with 
established  evaluation  factors.  See  EPSCO, 
Incorporated ,  supra . 291 
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CHAPTER  V 

CONCLUSION 

We  have  seen  that  Federal  Government  procurement 
officials  have  a  great  deal  of  discretion  throughout  the 
entire  source  selection  process.  That  discretion  is 
recognized  by  contracting  agencies  and  by  the  Comptroller 
General.  Yet,  in  spite  of  this  recognition,  protests 
against  competitive  source  selections  proliferate. 

In  the  overall  view  of  the  source  selection  process,  it 
would  seem  that  procuring  agencies  should  want  to  create  a 
selection  system  which  is,  of  course,  fair,  which  maximizes 
competition,  and  which  also  lessens  the  probability  of 
protests.  In  examining  the  many  protests  across  the 
spectrum  of  the  entire  source  selection  process,  two  things 
are  apparent. 

First,  ironically,  it  appears  that  the  more  objectivity 
present  in  the  selection  system,  the  more  likely  protests 
will  occur.  That  is,  given  the  discretion  of  the  source 
selection  official  in  making  his  final  decision,  as  the 
degree  of  objectivity  rises  in  the  evaluation  system, 
offerors  are  more  apt,  or  perhaps  possess  more  fertile 
grounds,  to  object  to  selection  decisions  which  differ  from, 
or  appear  to  differ  from,  evaluation  results.  Thus, 
protests  involving  all  aspects  of  numerical  formulae 
evaluations  abound.  Indeed,  notwithstanding  mccepted  agency 
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discretion,  it  is  not  an  easy  task  to  convince  an  offeror 
who  has  spent  a  significant  sum  of  money  just  preparing  an 
offer  that  all  of  those  elaborate,  sophisticated  evaluation 
procedures,  which  are  often  resplendent  with  highly 
qualified  evaluators,  should  not  really  be  binding. 
Nevertheless,  the  inherently  complex  nature  of  Government 
negotiated  procurements  certainly  requires  at  least  some 
form  or  degree  of  final  discretion.  In  establishing  a 
selection  system,  who  can  really  say  that  cost,  or 
responsibility-type  criteria  are  worth  20  points,  30  points, 
or  100  points?  Surely  it  can  be  accepted  that  Government 
officials  have  reason  and  intelligence  enough  to  decide 
whether  of  not  given  differences  in  estimated  costs  can 
outweigh  a  relatively  small  superiority  in  technical  merit. 
Some  procurements  may  lend  themselves  easily  to  objective 
evaluation  systems.  But  others,  particularly  major 
acquisitions,  may  fare  better  with  a  higher  degree  of 
subjectivity.  Perhaps  the  hir  Force  has  recognised  this  in 
their  likely  emphasis  on  a  color  coding  system  as  opposed  to 
numerical  formulae  in  the  proposed  revision  of  their  source 
selection  regulation,  as  noted  previously.  Ideally,  at 
least  at  the  major  factor  levels  of  major  acquisitions,  it 
would  appear  that  a  purely  subjective,  rational  system, 
applied  in  accordance  with  both  disclosed  criteria  and  a 
disclosed  evaluation  scheme,  might  provide  less  fertile 
grounds  for  protests  against  awards,  while  at  the  same  time 
maintain  the  requisite  fairness  and  competitiveness. 
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The  second  apparent  general  cause  of  many  protests 
concerns  the  elements  of  disclosure.  It  should  be  a 
fundamental  proposition  that  a  buyer  should  tell  the  seller 
exactly  what  he  wants  to  buy.  In  other  words,  to  Insure 
that  he  receives  a  product  which  will  best  satisfy  his 
needs,  the  buyer  should  be  as  specific  as  possible  in 
describing  those  needs  to  the  seller.  In  spite  of  what 
should  be  a  fundamental  proposition,  we  have  seen  that 
procuring  agencies  have  historically  been  reluctant  to 
disclose  many  aspects  of  their  selection  system,  which,  in 
essence,  will  determine  whether  or  not  the  product  is  in 
fact  appropriate.  As  noted,  only  two  agencies  surveyed 
recommended  disclosure  of  the  evaluation  method  to  be 
utilized.  A  classic  example  of  what  appears  to  be  a  rule 
without  a  valid  purpose  is  the  DAR's  prohibition  against 
disclosure  of  numerical  weights.  Logically,  it  would  seem 
that  by  revealing  those  weights,  offerors  would  be  able  to 
concentrate  their  efforts  in  the  most  important  areas  of  the 
procurement.  Purther,  disclosure  of  numerical  weights  might 
also  serve  to  reduce  protests.  Because  of  the  prohibition, 
offerors  normally  learn  of  the  precise  weights  during 
post-award  debriefings.  Under  the  Comptroller  General's  bid 
protest  rules,  protests  based  upon  alleged  defects  in  the 
solicitation  which  are  apparent  prior  to  the  closing  date 
for  receipt  of  initial  proposals  must  be  filed  prior  to  such 
closing  datet  otherwise,  protests  must  be  filed  not  later 
than  10  days  after  the  basis  for  the  protest ^becomes  known. 
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or  should  have  become  known,  whichever  ia  earlier.292  xf 
the  numerical  weights  were  disclosed  in  the  solicitation, 
protests  based  upon  those  weights  would  normally  be  denied 
as  untimely  if  filed  after  the  closing  date.  As  such,  the 
prospective  offeror  must  make  his  decision  during  the 
initial  stages  of  the  competition.  Also,  in  the  final 
analysis,  disclosure  of  those  weights  would  certainly  not  be 
detrimental  to  the  overall  integrity  of  the  procurement 
process.  Further,  the  recurrent  fear  that  agencies  will 
lose  their  discretion  if  offerors,  with  knowledge  of  precise 
weights,  are  able  to  "play"  with  the  system,  is  unfounded. 

We  have  already  seen  that  source  selection  officials  are  not 
bound  by  the  recommendations  or  findings  of  evaluation 
personnel,  as  long  as  the  selection  decision  has  a  rational 
basis. 

In  summary  then,  procuring  agencies  might  find  it 
worthwhile  to  inject  as  much  subjectivity  into  source 
evaluation  and  selection  systems  as  practicable,  and,  once 
that  system  is  chosen,  to  disclose  it  in  detail.  When  the 
process  is  complete,  the  requisite  rational  basis  for  the 
selection  decision  could  then  be  clearly  set  forth  in  a 
selection  document  for  the  edification  of  all  interested 
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(1974). 

184.  Dynalectron  Corporation,  Lockheed  Electronics  Company, 
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197.  DODD  4105.62,  supra,  note  48,  at  para.  IIIc ( 2 ) ( f ) ( 1 ) . 

198.  AFR  70-15,  supra,  note  4,  at  para.  3-2a(3). 
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200.  NAVMAT  INST.  4200.49,  supra,  note  8,  at  para.  7(a). 

201.  DARCOM  P  715-3,  supra,  note  199. 
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218.  52  Comp.  Gen.  161,  (1972)?  54  Comp.  Gen.  775  (1975). 
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225.  Id.,  at  3?  see  also,  H.  Bsmaili  4  Associates,  Inc., 
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Gen.  Dec.  B-187892,  August  7,  1977,  77-2  CPD  467. 

272.  Id. 

273.  Trilon  Educational  Corporation,  Comp.  Gen.  Dec. 
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